Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

[RFC] DNSRBL and variable settings

[RFC] DNSRBL and variable settings

2006-07-19 by Emmanuel Dreyfus

Hello

Following the talks about variables greylisting settings (greylist delay 
and timeout, autowhitelisting timeout) depending on the host:

Such a feature seems extremely interesting to me when combined with DNSRBL. 
One could have a greylisting delay of 45m by default, 1d for hosts listed
in a DNSRBL of xDSL pools, whitelist hosts listed in a DNSRBL of friendly 
networks and so on.

That would be the opportunity to add a blacklist feature

ACL entries would look like this: 
acl blacklist dnsrbl knownspammer.example.net
acl whitelist dnsrbl friends.example.net
acl greylist dnsrbl dslpool.example.net delay 1d timeout 3d autowhite 1d
acl greylist default delay 45m timeout 3d autowhite 1d

Another example, which is a bit less satisfying
acl blacklist dnsrbl knownspammer.example.net
acl whitelist dnsrbl friends.example.net
acl greylist rcpt john.doe@... dnsrbl dslpool.example.net delay 1d
acl greylist rcpt john.doe@... delay 45m
acl greylist rcpt manu@... dnsrbl dslpool.example.net delay 1d
acl greylist rcpt manu@... delay 45m
acl whitelist default

Problem: we have to duplicate a lot of informations. Obviously there is the
need for some factoring syntax. Any comment or idea?


-- 
Emmanuel Dreyfus
manu@...

RE: [milter-greylist] [RFC] DNSRBL and variable settings

2006-07-19 by Adri Koppes

Emmanuel,

This sounds very nice.
Especialy the dnsrbl whitelist/blacklist feature.

Adri.
Show quoted textHide quoted text
> -----Original Message-----
> From: milter-greylist@yahoogroups.com 
> [mailto:milter-greylist@yahoogroups.com] On Behalf Of Emmanuel Dreyfus
> Sent: woensdag 19 juli 2006 9:47
> To: milter-greylist@yahoogroups.com
> Subject: [milter-greylist] [RFC] DNSRBL and variable settings
> 
> Hello
> 
> Following the talks about variables greylisting settings 
> (greylist delay 
> and timeout, autowhitelisting timeout) depending on the host:
> 
> Such a feature seems extremely interesting to me when 
> combined with DNSRBL. 
> One could have a greylisting delay of 45m by default, 1d for 
> hosts listed
> in a DNSRBL of xDSL pools, whitelist hosts listed in a DNSRBL 
> of friendly 
> networks and so on.
> 
> That would be the opportunity to add a blacklist feature
> 
> ACL entries would look like this: 
> acl blacklist dnsrbl knownspammer.example.net
> acl whitelist dnsrbl friends.example.net
> acl greylist dnsrbl dslpool.example.net delay 1d timeout 3d 
> autowhite 1d
> acl greylist default delay 45m timeout 3d autowhite 1d
> 
> Another example, which is a bit less satisfying
> acl blacklist dnsrbl knownspammer.example.net
> acl whitelist dnsrbl friends.example.net
> acl greylist rcpt john.doe@... dnsrbl 
> dslpool.example.net delay 1d
> acl greylist rcpt john.doe@... delay 45m
> acl greylist rcpt manu@example.com dnsrbl dslpool.example.net delay 1d
> acl greylist rcpt manu@example.com delay 45m
> acl whitelist default
> 
> Problem: we have to duplicate a lot of informations. 
> Obviously there is the
> need for some factoring syntax. Any comment or idea?
> 
> 
> -- 
> Emmanuel Dreyfus
> manu@netbsd.org
> 
> 
> ------------------------ Yahoo! Groups Sponsor 
> --------------------~--> 
> See what's inside the new Yahoo! Groups email.
> http://us.click.yahoo.com/2pRQfA/bOaOAA/yQLSAA/W4wwlB/TM
> --------------------------------------------------------------
> ------~-> 
> 
>  
> Yahoo! Groups Links
> 
> 
> 
>  
> 
> 
>

Re: [milter-greylist] [RFC] DNSRBL and variable settings

2006-07-19 by Denis Solovyov

ED> Another example, which is a bit less satisfying
ED> acl blacklist dnsrbl knownspammer.example.net
ED> acl whitelist dnsrbl friends.example.net
ED> acl greylist rcpt john.doe@... dnsrbl dslpool.example.net delay 1d
ED> acl greylist rcpt john.doe@... delay 45m
ED> acl greylist rcpt manu@... dnsrbl dslpool.example.net delay 1d
ED> acl greylist rcpt manu@... delay 45m
ED> acl whitelist default
ED> Problem: we have to duplicate a lot of informations. Obviously there is the
ED> need for some factoring syntax. Any comment or idea?

Something like

acl greylist rcpt {
        john.doe@...,
        manu@...
} dnsrbl dslpool.example.net delay 1d

maybe?  I.e. let the user make lists of rcpts, addrs, and domains,
perhaps even named lists, such as:

list example rcpt { john.doe@..., manu@... }
acl greylist example dnsrbl dslpool.example.net delay 1d
acl greylist example delay 45m

It seems to me a bit similar to named.conf syntax.

"Consulting" DNSBLs is a great idea, but I think it would also be
the opportunity to implement some resource limit features, such as
the maximum possible greylisted/whitelisted tuples, since delaying
to DNSBLs (instead of just dropping connections) may increase the
number of recorded tuples dramatically.

P.S. IMHO blacklist acl is very much needed even without variable
greylisting...

Best regards,
Denis Solovyov

Re: [milter-greylist] [RFC] DNSRBL and variable settings

2006-07-19 by Michael Menge

Emmanuel Dreyfus wrote:
> Hello
> 
> Following the talks about variables greylisting settings (greylist delay 
> and timeout, autowhitelisting timeout) depending on the host:
> 
> Such a feature seems extremely interesting to me when combined with DNSRBL. 
> One could have a greylisting delay of 45m by default, 1d for hosts listed
> in a DNSRBL of xDSL pools, whitelist hosts listed in a DNSRBL of friendly 
> networks and so on.
> 
> That would be the opportunity to add a blacklist feature
> 
> ACL entries would look like this: 
> acl blacklist dnsrbl knownspammer.example.net
> acl whitelist dnsrbl friends.example.net
> acl greylist dnsrbl dslpool.example.net delay 1d timeout 3d autowhite 1d
> acl greylist default delay 45m timeout 3d autowhite 1d
> 
> Another example, which is a bit less satisfying
> acl blacklist dnsrbl knownspammer.example.net
> acl whitelist dnsrbl friends.example.net
> acl greylist rcpt john.doe@... dnsrbl dslpool.example.net delay 1d
> acl greylist rcpt john.doe@... delay 45m
> acl greylist rcpt manu@... dnsrbl dslpool.example.net delay 1d
> acl greylist rcpt manu@... delay 45m
> acl whitelist default
> 
> Problem: we have to duplicate a lot of informations. Obviously there is the
> need for some factoring syntax. Any comment or idea?
> 
> 
Grouping could reduce dublicate infos
i.e.

acl groupadd blafasel greylist dnsrbl dslpool.example.net delay 1d
acl groupadd blafasel greylist delay 45m
acl group blafasel rcpt john.doe@...
acl group blafasel rcpt manu@...

-- 
--------------------------------------------------------------------------------
M.Menge                                 Tel.: (49) 7071/29-70316
Universitaet Tuebingen                  Fax.: (49) 7071/29-5912
Zentrum fuer Datenverarbeitung          mail: menge@...-tuebingen.de
Waechterstrasse 76
72074 Tuebingen

Re: [RFC] DNSRBL and variable settings

2006-07-24 by manu@netbsd.org

Emmanuel Dreyfus <manu@...> wrote:

> Following the talks about variables greylisting settings (greylist delay
> and timeout, autowhitelisting timeout) depending on the host:

FWIW, I started working on per-acl greylisting and autowhitelisting
delays. That's a huge change, so please refrain from submitting a patch
in the upcoming days, as the merge will be a nightmare.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

milter-greylist 2.1.8 is available

2006-07-27 by manu@netbsd.org

Denis Solovyov <elk@...> wrote:

> P.S. IMHO blacklist acl is very much needed even without variable
> greylisting...

You have it!

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.8.tgz
MD5 (milter-greylist-2.1.8.tgz) = 839757e776eec20973dbfd6580450e55 

The only new feature since this afternoon's version is acl blacklist.

Please test things, the goal now is to relase milter-greylist 3.0 with
as much bugs fixed as possible. 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-27 by Michael Osten

On Jul 27, 2006, at 4:03 PM, manu@... wrote:

> Denis Solovyov <elk@...> wrote:
>
>> P.S. IMHO blacklist acl is very much needed even without variable
>> greylisting...
>
> You have it!
>
> http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.8.tgz
> MD5 (milter-greylist-2.1.8.tgz) = 839757e776eec20973dbfd6580450e55
>
> The only new feature since this afternoon's version is acl blacklist.
>
> Please test things, the goal now is to relase milter-greylist 3.0 with
> as much bugs fixed as possible.
>
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...


No changelog?
--
Michael Osten

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>>>> On Thu, 27 Jul 2006 23:03:40 +0200
>>>>> manu@... said:

manu> http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.8.tgz
manu> MD5 (milter-greylist-2.1.8.tgz) = 839757e776eec20973dbfd6580450e55 

manu> The only new feature since this afternoon's version is acl blacklist.

manu> Please test things, the goal now is to relase milter-greylist 3.0 with
manu> as much bugs fixed as possible. 

FreeBSD around 5.2-RELEASE and 6.1-STABLE don't have res_n*() but the
stock resolver is thread-safe.  FreeBSD 6.1-STABLE and later's stock
resolver is based on BIND9's one.

	FreeBSD		| thread-safe	res_n*()
	----------------+-----------------------
	< 5.2-R		| X		X
	5.2-R -- 6.1-R	| O		X
	> 6.1-S	 	| O		O

So, I wish to add --thread-safe-resolver option to configure script,
and make dnsrbl.c buildable without res_n*().

BTW, it seems you forgot to issue res_ndestroy() to release resources
used by resolver internally.  Slightly old BIND9 doesn't have
res_ndestroy().  If there is no res_ndestroy(), we need to issue
res_nclose() instead.

Here is the patch.  I've not tested it actually, yet.

Index: configure.ac
diff -u configure.ac.orig configure.ac
--- configure.ac.orig	Fri Jul 28 05:10:06 2006
+++ configure.ac	Fri Jul 28 13:13:45 2006
@@ -620,21 +620,24 @@
 AC_MSG_RESULT([$brokenpth])
 
 # Check is DNS resolver is re-entrant
-SAVEDCFLAGS=$CFLAGS
-CFLAGS=$CFLAGS" -Wall -Werror"
-AC_MSG_CHECKING([if DNS resolver is re-entrant])
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([
-		#include <netinet/in.h>
-		#include <arpa/inet.h>
-		#include <arpa/nameser.h>
-		#include <resolv.h>
-	],[
-		struct __res_state res;
+AC_ARG_WITH(thread-safe-resolver,
+	[  --with-thread-safe-resolver   Resolver is thread-safe],
+	[rdns=yes],[
+	SAVEDCFLAGS=$CFLAGS
+	CFLAGS=$CFLAGS" -Wall -Werror"
+	AC_MSG_CHECKING([if DNS resolver is re-entrant])
+	AC_COMPILE_IFELSE([AC_LANG_PROGRAM([
+			#include <netinet/in.h>
+			#include <arpa/inet.h>
+			#include <arpa/nameser.h>
+			#include <resolv.h>
+		],[
+			struct __res_state res;
 
-		res_ninit(&res);
-	])], [rdns=yes], [rdns=no])
-AC_MSG_RESULT([$rdns])
-CFLAGS=$SAVEDCFLAGS
+			res_ninit(&res);
+		])], [rdns=yes], [rdns=no])
+	AC_MSG_RESULT([$rdns])
+	CFLAGS=$SAVEDCFLAGS])
 
 # Check for DNSRBL
 AC_ARG_ENABLE(dnsrbl,
Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Wed Jul 26 22:26:02 2006
+++ dnsrbl.c	Fri Jul 28 13:13:24 2006
@@ -55,6 +55,23 @@ __RCSID("$Id: dnsrbl.c,v 1.3 2006/07/26 
 #include <arpa/nameser.h>
 #include <resolv.h>
 
+#ifndef NS_MAXMSG
+#define NS_MAXMSG	65535
+#endif
+
+#ifdef res_ninit
+#define HAVE_RESN	1
+#ifndef res_ndestroy
+#define res_ndestroy(res)	res_nclose(res)
+#endif
+#else
+#define	res_ninit(res) \
+	((_res.options & RES_INIT) == 0 && res_init())
+#define res_nquery(res, req, class, type, ans, anslen)	\
+	res_query(req, class, type, ans, anslen)
+#define res_ndestroy(res)
+#endif
+
 #include "milter-greylist.h"
 #include "dnsrbl.h"
 
@@ -68,14 +85,16 @@ void
 dnsrbl_init(void) {
 	LIST_INIT(&dnsrbl_head);
 	return;
-}  
+}
 
 int
 dnsrbl_check_source(sa, source)
 	struct sockaddr *sa;
         struct dnsrbl_entry *source;
 {
+#ifdef HAVE_RESN
 	struct __res_state res;
+#endif
 	struct sockaddr_storage ss;
 	char req[NS_MAXDNAME + 1];
 	char ans[NS_MAXMSG + 1];
@@ -86,6 +105,7 @@ dnsrbl_check_source(sa, source)
 	int i;
 	char *dnsrbl = source->d_domain;
 	struct sockaddr *blacklisted;
+	int retval = 0;
 
 	blacklisted = (struct sockaddr *)&source->d_blacklisted;
 
@@ -110,7 +130,8 @@ dnsrbl_check_source(sa, source)
 
 	if ((inet_ntop(ss.ss_family, addr, req, NS_MAXDNAME)) == NULL){
 		syslog(LOG_ERR, "inet_ntop failed: %s", strerror(errno));
-		return -1;
+		retval = -1;
+		goto end;
 	}
 
 	(void)strncat(req, ".", NS_MAXDNAME);
@@ -118,13 +139,14 @@ dnsrbl_check_source(sa, source)
 
 	anslen = res_nquery(&res, req, C_IN, T_A, ans, sizeof(ans));
 	if (anslen == -1)
-		return 0;
+		goto end;
 
 	if (ns_initparse(ans, anslen, &handle) < 0) {
 		syslog(LOG_ERR, "ns_initparse failed: %s", strerror(errno));
-		return -1;
+		retval = -1;
+		goto end;
 	}
-	
+
 	for (i = 0; i < ns_msg_count(handle, ns_s_an); i++) {
 		char *addr;
 		size_t len;
@@ -132,7 +154,8 @@ dnsrbl_check_source(sa, source)
 		if ((ns_parserr(&handle, ns_s_an, i, &rr)) != 0) {
 			syslog(LOG_ERR, "ns_parserr failed: %s", 
 			    strerror(errno));
-			return -1;
+			retval = -1;
+			goto end;
 		}
 
 		switch (blacklisted->sa_family) {
@@ -167,11 +190,15 @@ dnsrbl_check_source(sa, source)
 			break;
 		}
 
-		if (memcmp(addr, rr.rdata, len) == 0)
-			return 1;
+		if (memcmp(addr, rr.rdata, len) == 0) {
+			retval = 1;
+			goto end;
+		}
 	}
 
-	return 0;
+end:
+	res_ndestroy(&res);
+	return retval;
 }
 
 

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by manu@netbsd.org

Hajimu UMEMOTO <ume@...> wrote:

> So, I wish to add --thread-safe-resolver option to configure script,
> and make dnsrbl.c buildable without res_n*().

Checked in. Thank you for the input.
I'll wait a bit before releasing 2.1.9, so that we don't have a release
every 8 hour :-)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Denis Solovyov

>> P.S. IMHO blacklist acl is very much needed even without variable
>> greylisting...
mno> You have it!
mno> The only new feature since this afternoon's version is acl blacklist.

Just one wish more.

Blacklisting  is  much  more  serious thing than greylisting, so I think
there should be a directive to set a user-defined "go away" message with
a  specific  explanation  on  why  one is blacklisted. As for me I would
appreciate at least one global "go away" custom message.  Example:

goawaymsg "You are blacklisted by our antispam software - please see http://some.url/antispam-policy"

Best regards,
Denis Solovyov

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 07:18:33 +0200
>>>>> manu@... said:

manu> Checked in. Thank you for the input.
manu> I'll wait a bit before releasing 2.1.9, so that we don't have a release
manu> every 8 hour :-)

Thank you.
I made one more patch.  Since sa_len is referred, I think it is not
buildable on Linux.  And, some cleanups.
I've not test it actually, yet.

Index: acl.c
diff -u -p acl.c.orig acl.c
--- acl.c.orig	Fri Jul 28 05:08:32 2006
+++ acl.c	Fri Jul 28 16:00:18 2006
@@ -615,7 +615,7 @@ acl_filter(sa, salen, hostname, from, rc
 		}
 
 		if (acl->a_dnsrbl != NULL) {
-			if (dnsrbl_check_source(sa, acl->a_dnsrbl) != 0) {
+			if (dnsrbl_check_source(sa, salen, acl->a_dnsrbl) != 0) {
 				retval |= EXF_DNSRBL;
 				if (conf.c_debug) {
 					iptostring(sa, salen, 
Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Fri Jul 28 13:18:56 2006
+++ dnsrbl.c	Fri Jul 28 16:30:26 2006
@@ -73,6 +73,7 @@ __RCSID("$Id: dnsrbl.c,v 1.3 2006/07/26 
 #endif
 
 #include "milter-greylist.h"
+#include "pending.h"
 #include "dnsrbl.h"
 
 /* 
@@ -88,8 +89,9 @@ dnsrbl_init(void) {
 }
 
 int
-dnsrbl_check_source(sa, source)
+dnsrbl_check_source(sa, salen, source)
 	struct sockaddr *sa;
+	socklen_t salen;
         struct dnsrbl_entry *source;
 {
 #ifdef HAVE_RESN
@@ -99,7 +101,6 @@ dnsrbl_check_source(sa, source)
 	char req[NS_MAXDNAME + 1];
 	char ans[NS_MAXMSG + 1];
 	int anslen;
-	void *addr;
 	ns_msg handle;
 	ns_rr rr;
 	int i;
@@ -107,29 +108,14 @@ dnsrbl_check_source(sa, source)
 	struct sockaddr *blacklisted;
 	int retval = 0;
 
-	blacklisted = (struct sockaddr *)&source->d_blacklisted;
+	blacklisted = SA(&source->d_blacklisted);
 
 	res_ninit(&res);
 
-	reverse_endian((struct sockaddr *)&ss, sa);
+	reverse_endian(SA(&ss), sa, salen);
 
-	switch (ss.ss_family) {
-	case AF_INET:
-		addr = &((struct sockaddr_in *)&ss)->sin_addr;
-		break;
-#ifdef AF_INET6
-	case AF_INET6:
-		addr = &((struct sockaddr_in6 *)&ss)->sin6_addr;
-		break;
-#endif
-	default:
-		syslog(LOG_ERR, "unexpected address family %d", ss.ss_family);
-		exit(EX_SOFTWARE);
-		break;
-	}
-
-	if ((inet_ntop(ss.ss_family, addr, req, NS_MAXDNAME)) == NULL){
-		syslog(LOG_ERR, "inet_ntop failed: %s", strerror(errno));
+	if ((iptostring(SA(&ss), salen, req, NS_MAXDNAME)) == NULL) {
+		syslog(LOG_ERR, "iptostring failed: %s", strerror(errno));
 		retval = -1;
 		goto end;
 	}
@@ -204,25 +190,24 @@ end:
 
 /* XXX this code is probably broken with IPv6 */
 void
-reverse_endian(dst, src)
+reverse_endian(dst, src, len)
 	struct sockaddr *src;
 	struct sockaddr *dst;
+	socklen_t len;
 {
-	int i, len;
+	int i;
 	char *src_start;
 	char *dst_start;
 
 	switch (src->sa_family) {
 	case AF_INET:
-		src_start = (char *)&((struct sockaddr_in *)src)->sin_addr;
-		dst_start = (char *)&((struct sockaddr_in *)dst)->sin_addr;
-		len = sizeof(((struct sockaddr_in *)src)->sin_addr);
+		src_start = (char *)SADDR4(src);
+		dst_start = (char *)SADDR4(dst);
 		break;
 #ifdef AF_INET6
 	case AF_INET6:
-		src_start = (char *)&((struct sockaddr_in6 *)src)->sin6_addr;
-		dst_start = (char *)&((struct sockaddr_in6 *)dst)->sin6_addr;
-		len = sizeof(((struct sockaddr_in6 *)src)->sin6_addr);
+		src_start = (char *)SADDR6(src);
+		dst_start = (char *)SADDR6(dst);
 		break;
 #endif
 	default:
@@ -232,7 +217,9 @@ reverse_endian(dst, src)
 	}
 
 	dst->sa_family = src->sa_family;
+#ifdef HAVE_SA_LEN
 	dst->sa_len = src->sa_len;
+#endif
 
 	for (i = 0; i < len; i++)
 		dst_start[len - 1 - i] = src_start[i];
@@ -247,17 +234,34 @@ dnsrbl_source_add(name, domain, blacklis
 	struct sockaddr *blacklisted;
 {
 	struct dnsrbl_entry *de;
+	socklen_t len;
 
 	if ((de = malloc(sizeof(*de))) == NULL) {
 		syslog(LOG_ERR, "malloc failed: %s", strerror(errno));
 		exit(EX_OSERR);
 	}
 
+	switch (blacklisted->sa_family) {
+	case AF_INET:
+		len = sizeof(*SADDR4(blacklisted));
+		break;
+#ifdef AF_INET6
+	case AF_INET6:
+		len = sizeof(*SADDR6(blacklisted));
+		break;
+#endif
+	default:
+		syslog(LOG_ERR, "invalid address family %d",
+		    blacklisted->sa_family);
+		exit(EX_SOFTWARE);
+		break;
+	}
+
 	strncpy(de->d_name, name, sizeof(de->d_name));
 	de->d_name[sizeof(de->d_name) - 1] = '\0';
 	strncpy(de->d_domain, domain, sizeof(de->d_domain));
 	de->d_domain[sizeof(de->d_domain) - 1] = '\0';
-	memcpy(&de->d_blacklisted, blacklisted, blacklisted->sa_len);
+	memcpy(&de->d_blacklisted, blacklisted, len);
 
 	LIST_INSERT_HEAD(&dnsrbl_head, de, d_list);
 
Index: dnsrbl.h
diff -u dnsrbl.h.orig dnsrbl.h
--- dnsrbl.h.orig	Thu Jul 27 21:48:24 2006
+++ dnsrbl.h	Fri Jul 28 16:12:53 2006
@@ -45,8 +45,8 @@
 };
 
 void dnsrbl_init(void);
-int dnsrbl_check_source(struct sockaddr *, struct dnsrbl_entry *);
-void reverse_endian(struct sockaddr *, struct sockaddr *);
+int dnsrbl_check_source(struct sockaddr *, socklen_t, struct dnsrbl_entry *);
+void reverse_endian(struct sockaddr *, struct sockaddr *, socklen_t);
 void dnsrbl_source_add(char *, char *, struct sockaddr *);
 struct dnsrbl_entry *dnsrbl_byname(char *);
 void dnsrbl_clear(void);

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>> Fri, 28 Jul 2006 16:47:39 +0900,
>>> Hajimu UMEMOTO <ume@...> said:

ume> I made one more patch.  Since sa_len is referred, I think it is not
ume> buildable on Linux.  And, some cleanups.
ume> I've not test it actually, yet.

Oops, it was wrong.  Please throw my previous patch away and use this
patch instead.  It seems working with this patch, here.

BTW, in current implementation, IPv6 address is also handled as same
manner as IPv4 address.  Perhaps, it is wrong.  But, I have no idea
how it should be handled.  Are there any specification around here for
IPv6?

Index: acl.c
diff -u -p acl.c.orig acl.c
--- acl.c.orig	Fri Jul 28 05:08:32 2006
+++ acl.c	Fri Jul 28 16:00:18 2006
@@ -615,7 +615,7 @@ acl_filter(sa, salen, hostname, from, rc
 		}
 
 		if (acl->a_dnsrbl != NULL) {
-			if (dnsrbl_check_source(sa, acl->a_dnsrbl) != 0) {
+			if (dnsrbl_check_source(sa, salen, acl->a_dnsrbl) != 0) {
 				retval |= EXF_DNSRBL;
 				if (conf.c_debug) {
 					iptostring(sa, salen, 
Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Fri Jul 28 13:18:56 2006
+++ dnsrbl.c	Fri Jul 28 20:06:24 2006
@@ -73,6 +73,7 @@ __RCSID("$Id: dnsrbl.c,v 1.3 2006/07/26 
 #endif
 
 #include "milter-greylist.h"
+#include "pending.h"
 #include "dnsrbl.h"
 
 /* 
@@ -88,8 +89,9 @@ dnsrbl_init(void) {
 }
 
 int
-dnsrbl_check_source(sa, source)
+dnsrbl_check_source(sa, salen, source)
 	struct sockaddr *sa;
+	socklen_t salen;
         struct dnsrbl_entry *source;
 {
 #ifdef HAVE_RESN
@@ -99,7 +101,6 @@ dnsrbl_check_source(sa, source)
 	char req[NS_MAXDNAME + 1];
 	char ans[NS_MAXMSG + 1];
 	int anslen;
-	void *addr;
 	ns_msg handle;
 	ns_rr rr;
 	int i;
@@ -107,29 +108,14 @@ dnsrbl_check_source(sa, source)
 	struct sockaddr *blacklisted;
 	int retval = 0;
 
-	blacklisted = (struct sockaddr *)&source->d_blacklisted;
+	blacklisted = SA(&source->d_blacklisted);
 
 	res_ninit(&res);
 
-	reverse_endian((struct sockaddr *)&ss, sa);
+	reverse_endian(SA(&ss), sa);
 
-	switch (ss.ss_family) {
-	case AF_INET:
-		addr = &((struct sockaddr_in *)&ss)->sin_addr;
-		break;
-#ifdef AF_INET6
-	case AF_INET6:
-		addr = &((struct sockaddr_in6 *)&ss)->sin6_addr;
-		break;
-#endif
-	default:
-		syslog(LOG_ERR, "unexpected address family %d", ss.ss_family);
-		exit(EX_SOFTWARE);
-		break;
-	}
-
-	if ((inet_ntop(ss.ss_family, addr, req, NS_MAXDNAME)) == NULL){
-		syslog(LOG_ERR, "inet_ntop failed: %s", strerror(errno));
+	if ((iptostring(SA(&ss), salen, req, NS_MAXDNAME)) == NULL) {
+		syslog(LOG_ERR, "iptostring failed: %s", strerror(errno));
 		retval = -1;
 		goto end;
 	}
@@ -165,7 +151,7 @@ dnsrbl_check_source(sa, source)
 			if (rr.type != T_A)
 				continue;
 
-			sin = (struct sockaddr_in *)blacklisted;
+			sin = SA4(blacklisted);
 			addr = (char *)&sin->sin_addr;
 			len = sizeof(sin->sin_addr);
 			break;
@@ -177,7 +163,7 @@ dnsrbl_check_source(sa, source)
 			if (rr.type != T_AAAA)
 				continue;
 
-			sin6 = (struct sockaddr_in6 *)blacklisted;
+			sin6 = SA6(blacklisted);
 			addr = (char *)&sin6->sin6_addr;
 			len = sizeof(sin6->sin6_addr);
 			break;
@@ -214,15 +200,15 @@ reverse_endian(dst, src)
 
 	switch (src->sa_family) {
 	case AF_INET:
-		src_start = (char *)&((struct sockaddr_in *)src)->sin_addr;
-		dst_start = (char *)&((struct sockaddr_in *)dst)->sin_addr;
-		len = sizeof(((struct sockaddr_in *)src)->sin_addr);
+		src_start = (char *)SADDR4(src);
+		dst_start = (char *)SADDR4(dst);
+		len = sizeof(*SADDR4(src));
 		break;
 #ifdef AF_INET6
 	case AF_INET6:
-		src_start = (char *)&((struct sockaddr_in6 *)src)->sin6_addr;
-		dst_start = (char *)&((struct sockaddr_in6 *)dst)->sin6_addr;
-		len = sizeof(((struct sockaddr_in6 *)src)->sin6_addr);
+		src_start = (char *)SADDR6(src);
+		dst_start = (char *)SADDR6(dst);
+		len = sizeof(*SADDR6(src));
 		break;
 #endif
 	default:
@@ -232,7 +218,9 @@ reverse_endian(dst, src)
 	}
 
 	dst->sa_family = src->sa_family;
+#ifdef HAVE_SA_LEN
 	dst->sa_len = src->sa_len;
+#endif
 
 	for (i = 0; i < len; i++)
 		dst_start[len - 1 - i] = src_start[i];
@@ -247,17 +235,34 @@ dnsrbl_source_add(name, domain, blacklis
 	struct sockaddr *blacklisted;
 {
 	struct dnsrbl_entry *de;
+	socklen_t salen;
 
 	if ((de = malloc(sizeof(*de))) == NULL) {
 		syslog(LOG_ERR, "malloc failed: %s", strerror(errno));
 		exit(EX_OSERR);
 	}
 
+	switch (blacklisted->sa_family) {
+	case AF_INET:
+		salen = sizeof(struct sockaddr_in);
+		break;
+#ifdef AF_INET6
+	case AF_INET6:
+		salen = sizeof(struct sockaddr_in6);
+		break;
+#endif
+	default:
+		syslog(LOG_ERR, "invalid address family %d",
+		    blacklisted->sa_family);
+		exit(EX_SOFTWARE);
+		break;
+	}
+
 	strncpy(de->d_name, name, sizeof(de->d_name));
 	de->d_name[sizeof(de->d_name) - 1] = '\0';
 	strncpy(de->d_domain, domain, sizeof(de->d_domain));
 	de->d_domain[sizeof(de->d_domain) - 1] = '\0';
-	memcpy(&de->d_blacklisted, blacklisted, blacklisted->sa_len);
+	memcpy(&de->d_blacklisted, blacklisted, salen);
 
 	LIST_INSERT_HEAD(&dnsrbl_head, de, d_list);
 
Index: dnsrbl.h
diff -u dnsrbl.h.orig dnsrbl.h
--- dnsrbl.h.orig	Thu Jul 27 21:48:24 2006
+++ dnsrbl.h	Fri Jul 28 20:02:01 2006
@@ -45,7 +45,7 @@
 };
 
 void dnsrbl_init(void);
-int dnsrbl_check_source(struct sockaddr *, struct dnsrbl_entry *);
+int dnsrbl_check_source(struct sockaddr *, socklen_t, struct dnsrbl_entry *);
 void reverse_endian(struct sockaddr *, struct sockaddr *);
 void dnsrbl_source_add(char *, char *, struct sockaddr *);
 struct dnsrbl_entry *dnsrbl_byname(char *);


Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by manu@netbsd.org

Denis Solovyov <elk@...> wrote:

> Blacklisting  is  much  more  serious thing than greylisting, so I think
> there should be a directive to set a user-defined "go away" message with
> a  specific  explanation  on  why  one is blacklisted. As for me I would
> appreciate at least one global "go away" custom message.  Example:
> 
> goawaymsg "You are blacklisted by our antispam software - please see
> http://some.url/antispam-policy"

Sure, but the next item on my todo list is the multi-line list
statements. Feel free to submit a patch for your change, however.

The setting could be per-ACL, I'm sure it could be useful.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>> Fri, 28 Jul 2006 20:23:10 +0900,
>>> Hajimu UMEMOTO <ume@...> said:

ume> Oops, it was wrong.  Please throw my previous patch away and use this
ume> patch instead.  It seems working with this patch, here.

Umm, it seems the reson string in X-Greylist: header is strange:

	X-Greylist: Internal error , not delayed by milter-greylist-2.1.8
	 (ameno.mahoroba.org [218.45.22.175]); Fri, 28 Jul 2006 20:32:00
	 +0900 (JST)

I'm testing it with following configuration:

	dnsrbl "MAHOROBA DNSWL" dnswl.mahoroba.org 127.0.0.10
	acl whitelist dnsrbl "MAHOROBA DNSWL"

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re[2]: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Denis Solovyov

>> a  specific  explanation  on  why  one is blacklisted. As for me I would
>> appreciate at least one global "go away" custom message.  Example:
>> goawaymsg "You are blacklisted by our antispam software - please see
>> http://some.url/antispam-policy"
mno> Sure, but the next item on my todo list is the multi-line list
mno> statements. Feel free to submit a patch for your change, however.

Unfortunately I'm not good enough in programming. All I can do is to
change "Go away!" string in milter-greylist.c to what I want (as a
temporary solution suitable for myself only of course).

Maybe someone else implements this idea?

mno> The setting could be per-ACL, I'm sure it could be useful.

There should be one custom default setting, and per-acl optional
settings which will override the default.

Best regards,
Denis Solovyov

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Emmanuel Dreyfus

On Fri, Jul 28, 2006 at 08:54:56PM +0900, Hajimu UMEMOTO wrote:
> Umm, it seems the reson string in X-Greylist: header is strange:
> 
> 	X-Greylist: Internal error , not delayed by milter-greylist-2.1.8
> 	 (ameno.mahoroba.org [218.45.22.175]); Fri, 28 Jul 2006 20:32:00
> 	 +0900 (JST)
> 
> I'm testing it with following configuration:
> 
> 	dnsrbl "MAHOROBA DNSWL" dnswl.mahoroba.org 127.0.0.10
> 	acl whitelist dnsrbl "MAHOROBA DNSWL"

At mine the milter is unstable (2.1.8 plus your patches. It was also unstable
at 2.1.8). I think there is a corrupted memory issue. Here is a header that
was generated:
Sender IP whitelisted, not delayed by milter-greylist-2.1.9a1 (mx2.espci.fr [193.54.82.18]); Fri, 28 Jul 2006 16:53:17 +0200 (\232\265(\001\325KpA\226z\210A\211\202\024\001\325Q\253\001\325Q\020A\20710 )

Reading the code, it's not the string generation, it must be more widespread.

If you find something wrong...

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 14:58:41 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Fri, Jul 28, 2006 at 08:54:56PM +0900, Hajimu UMEMOTO wrote:
> Umm, it seems the reson string in X-Greylist: header is strange:
> 
> 	X-Greylist: Internal error , not delayed by milter-greylist-2.1.8
> 	 (ameno.mahoroba.org [218.45.22.175]); Fri, 28 Jul 2006 20:32:00
> 	 +0900 (JST)
> 
> I'm testing it with following configuration:
> 
> 	dnsrbl "MAHOROBA DNSWL" dnswl.mahoroba.org 127.0.0.10
> 	acl whitelist dnsrbl "MAHOROBA DNSWL"

I found the cause.  The reason strings were not set.  Here is a patch:

Index: acl.c
diff -u -p acl.c.orig acl.c
--- acl.c.orig	Fri Jul 28 05:08:32 2006
+++ acl.c	Sat Jul 29 00:12:06 2006
@@ -691,6 +691,12 @@ acl_filter(sa, salen, hostname, from, rc
 			iptostring(sa, salen, addrstr, sizeof(addrstr));
 			snprintf(tmpstr, sizeof(tmpstr),
 			     "address %s is whitelisted", addrstr);
+			ADD_REASON(whystr, tmpstr);
+		}
+		if (retval & EXF_DNSRBL) {
+			iptostring(sa, salen, addrstr, sizeof(addrstr));
+			snprintf(tmpstr, sizeof(tmpstr),
+			     "address %s is whitelisted by DNSRBL", addrstr);
 			ADD_REASON(whystr, tmpstr);
 		}
 		if (retval & EXF_DOMAIN) {
Index: milter-greylist.c
diff -u -p milter-greylist.c.orig milter-greylist.c
--- milter-greylist.c.orig	Fri Jul 28 05:08:32 2006
+++ milter-greylist.c	Sat Jul 29 00:23:20 2006
@@ -572,6 +572,10 @@ mlfi_eom(ctx)
 			ADD_REASON(whystr, "IP, sender and recipient auto-whitelisted");
 			priv->priv_whitelist &= ~EXF_AUTO;
 		}
+		if (priv->priv_whitelist & EXF_DNSRBL) {
+			ADD_REASON(whystr, "Sender IP whitelisted by DNSRBL");
+			priv->priv_whitelist &= ~EXF_DNSRBL;
+		}
 		if (priv->priv_whitelist & EXF_DEFAULT) {
 			ADD_REASON(whystr, "Default is to whitelist mail");
 			priv->priv_whitelist &= ~EXF_DEFAULT;


manu> At mine the milter is unstable (2.1.8 plus your patches. It was also unstable
manu> at 2.1.8). I think there is a corrupted memory issue. Here is a header that
manu> was generated:
manu> Sender IP whitelisted, not delayed by milter-greylist-2.1.9a1 (mx2.espci.fr [193.54.82.18]); Fri, 28 Jul 2006 16:53:17 +0200 (\232\265(\001\325KpA\226z\210A\211\202\024\001\325Q\253\001\325Q\020A\20710 )

manu> Reading the code, it's not the string generation, it must be more widespread.

Perhaps, it is another issue I didn't met yet.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Emmanuel Dreyfus

On Fri, Jul 28, 2006 at 08:23:10PM +0900, Hajimu UMEMOTO wrote:
> Oops, it was wrong.  Please throw my previous patch away and use this
> patch instead.  It seems working with this patch, here.

Checked in, with a few additionnal changes so that it builds...
I'll roll out 2.1.9 so that everyone can test with your patches easily.

> BTW, in current implementation, IPv6 address is also handled as same
> manner as IPv4 address.  Perhaps, it is wrong.  But, I have no idea
> how it should be handled.  Are there any specification around here for
> IPv6?

I'm not aware of any IPv6 DNSRBL...

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Emmanuel Dreyfus

On Sat, Jul 29, 2006 at 12:37:35AM +0900, Hajimu UMEMOTO wrote:
> > 	dnsrbl "MAHOROBA DNSWL" dnswl.mahoroba.org 127.0.0.10
> > 	acl whitelist dnsrbl "MAHOROBA DNSWL"

What is this whitelist, BTW?

-- 
Emmanuel Dreyfus
manu@...

milter-greylist 2.1.9 is available

2006-07-28 by Emmanuel Dreyfus

Here is it:

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.9.tgz
MD5 (milter-greylist-2.1.9.tgz) = fae68cc6e036e9ff6c00e3a4ccf612dc

This is just 2.1.8 with a few fixes from Hajimu UMEMOTO. It's still unstable
at mine. I suspec thread-safety issues with the DNS, although I'm linking
with BIND9's libbind...

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 15:39:38 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> Checked in, with a few additionnal changes so that it builds...

It seems my patch was incomplete.  Sorry for the mess.

manu> I'll roll out 2.1.9 so that everyone can test with your patches easily.

Thank you.  It seems running well, here.

When blacklisted holds an IPv6 address, we need to query AAAA RR
instead of A RR.  Here is a patch with some cleanup.

Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Sat Jul 29 00:41:51 2006
+++ dnsrbl.c	Sat Jul 29 03:42:14 2006
@@ -103,10 +103,12 @@ dnsrbl_check_source(sa, salen, source)
 	int anslen;
 	ns_msg handle;
 	ns_rr rr;
-	int i;
+	int qtype, i;
 	char *dnsrbl = source->d_domain;
 	struct sockaddr *blacklisted;
 	int retval = 0;
+	char *addr;
+	size_t len;
 
 	blacklisted = SA(&source->d_blacklisted);
 
@@ -123,7 +125,27 @@ dnsrbl_check_source(sa, salen, source)
 	(void)strncat(req, ".", NS_MAXDNAME);
 	(void)strncat(req, dnsrbl, NS_MAXDNAME);
 
-	anslen = res_nquery(&res, req, C_IN, T_A, ans, sizeof(ans));
+	switch (blacklisted->sa_family) {
+	case AF_INET:
+		qtype = T_A;
+		addr = (char *)SADDR4(blacklisted);
+		len = sizeof(*SADDR4(blacklisted));
+		break;
+#ifdef AF_INET6
+	case AF_INET6:
+		qtype = T_AAAA;
+		addr = (char *)SADDR6(blacklisted);
+		len = sizeof(*SADDR6(blacklisted));
+		break;
+#endif
+	default:
+		syslog(LOG_ERR, "unexpected address family %d",
+		    blacklisted->sa_family);
+		exit(EX_SOFTWARE);
+		break;
+	}
+
+	anslen = res_nquery(&res, req, C_IN, qtype, ans, sizeof(ans));
 	if (anslen == -1)
 		goto end;
 
@@ -134,9 +156,6 @@ dnsrbl_check_source(sa, salen, source)
 	}
 
 	for (i = 0; i < ns_msg_count(handle, ns_s_an); i++) {
-		char *addr;
-		size_t len;
-
 		if ((ns_parserr(&handle, ns_s_an, i, &rr)) != 0) {
 			syslog(LOG_ERR, "ns_parserr failed: %s", 
 			    strerror(errno));
@@ -145,35 +164,16 @@ dnsrbl_check_source(sa, salen, source)
 		}
 
 		switch (blacklisted->sa_family) {
-		case AF_INET: {
-			struct sockaddr_in *sin;
-
+		case AF_INET:
 			if (rr.type != T_A)
 				continue;
-
-			sin = SA4(blacklisted);
-			addr = (char *)&sin->sin_addr;
-			len = sizeof(sin->sin_addr);
 			break;
-		}
 #ifdef AF_INET6
-		case AF_INET6: {
-			struct sockaddr_in6 *sin6;
-
+		case AF_INET6:
 			if (rr.type != T_AAAA)
 				continue;
-
-			sin6 = SA6(blacklisted);
-			addr = (char *)&sin6->sin6_addr;
-			len = sizeof(sin6->sin6_addr);
 			break;
-		}
 #endif
-		default:
-			syslog(LOG_ERR, "unexpected address family %d", 
-			    blacklisted->sa_family);
-			exit(EX_SOFTWARE);
-			break;
 		}
 
 		if (memcmp(addr, rr.rdata, len) == 0) {


> BTW, in current implementation, IPv6 address is also handled as same
> manner as IPv4 address.  Perhaps, it is wrong.  But, I have no idea
> how it should be handled.  Are there any specification around here for
> IPv6?

manu> I'm not aware of any IPv6 DNSRBL...

Okay.  Then, I think we have two choices:

  1) disable DNSRBL query against an IPv6 address
  2) make our format for an IPv6 address

If we choose 2), I offer to use similar format with IPv6 PTR RR like:

  0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.example.com.

If you like it, I'll implement it.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 15:51:18 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Sat, Jul 29, 2006 at 12:37:35AM +0900, Hajimu UMEMOTO wrote:
> > 	dnsrbl "MAHOROBA DNSWL" dnswl.mahoroba.org 127.0.0.10
> > 	acl whitelist dnsrbl "MAHOROBA DNSWL"

manu> What is this whitelist, BTW?

It is my local server for testing.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-28 by manu@netbsd.org

Hajimu UMEMOTO <ume@...> wrote:

> Okay.  Then, I think we have two choices:
> 
>   1) disable DNSRBL query against an IPv6 address
>   2) make our format for an IPv6 address
> 
> If we choose 2), I offer to use similar format with IPv6 PTR RR like:
> 
>   0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.example.com.
> 
> If you like it, I'll implement it.

Well, what's the point of making a format, if we have no garantee that
people will follow the same path? I'd be in favor of handling IPv6 the
day it becomes useful and specified.

What do you think?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-29 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 14:58:41 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> At mine the milter is unstable (2.1.8 plus your patches. It was also unstable
manu> at 2.1.8). I think there is a corrupted memory issue. Here is a header that
manu> was generated:
manu> Sender IP whitelisted, not delayed by milter-greylist-2.1.9a1 (mx2.espci.fr [193.54.82.18]); Fri, 28 Jul 2006 16:53:17 +0200 (\232\265(\001\325KpA\226z\210A\211\202\024\001\325Q\253\001\325Q\020A\20710 )

manu> Reading the code, it's not the string generation, it must be more widespread.

manu> If you find something wrong...

I'm not sure if this patch solves your problem, it seems res must be
initialized before calling memset().

Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Sat Jul 29 00:41:51 2006
+++ dnsrbl.c	Sun Jul 30 04:27:23 2006
@@ -103,6 +103,9 @@ dnsrbl_check_source(sa, salen, source)
 
 	blacklisted = SA(&source->d_blacklisted);
 
+#ifdef HAVE_RESN
+	memset(&res, 0, sizeof(res));
+#endif
 	res_ninit(&res);
 
 	reverse_endian(SA(&ss), sa);


Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-29 by manu@netbsd.org

Hajimu UMEMOTO <ume@...> wrote:

> I'm not sure if this patch solves your problem, it seems res must be
> initialized before calling memset().

You mean res_ninit(), I guess.

I'll try this on monday.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-31 by Hajimu UMEMOTO

Hi,

>>>>> On Fri, 28 Jul 2006 22:54:51 +0200
>>>>> manu@... said:

manu> Hajimu UMEMOTO <ume@...> wrote:

> Okay.  Then, I think we have two choices:
> 
>   1) disable DNSRBL query against an IPv6 address
>   2) make our format for an IPv6 address
> 
> If we choose 2), I offer to use similar format with IPv6 PTR RR like:
> 
>   0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.example.com.
> 
> If you like it, I'll implement it.

manu> Well, what's the point of making a format, if we have no garantee that
manu> people will follow the same path? I'd be in favor of handling IPv6 the
manu> day it becomes useful and specified.

manu> What do you think?

AFAIK, there is no standard nor de facto to handle an IPv6 address by
DNSRBL.  Further, there are no available service which supports an
IPv6 address.  However, if someone want to serve the server, it will
be possible, as far as there is certain format for an IPv6 address.
In current implementation, the form is

	inet_ntop(reverse(address)) + domain

even for an IPv6 address.  Since inet_ntop produces a canonical form
of IPv6 address representation, it is hard to specify A RR in their
zone file.  Further, it contains `:' rather than `.'.  This is why I
offer an IPv6 PTR RR like format.
Even if it will not be a de facto, still we will be able to use this
format in anyway.  However, if DNSRBL will incorporate another format
in their service, we will need to chase the format.  Perhaps, we can
support both format with a configure option.
If we don't want to have a compatibility issue in the future, we
should disable a DNSRBL query against an IPv6 address for now.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] milter-greylist 2.1.8 is available

2006-07-31 by manu@netbsd.org

Hajimu UMEMOTO <ume@...> wrote:

> Even if it will not be a de facto, still we will be able to use this
> format in anyway.  However, if DNSRBL will incorporate another format
> in their service, we will need to chase the format.  Perhaps, we can
> support both format with a configure option.
> If we don't want to have a compatibility issue in the future, we
> should disable a DNSRBL query against an IPv6 address for now.

I think we should do that, and re-introduce IPv6 support the day an IPv6
DNSRBL appear. For now the IPv6 code would not be useful for a single
user in the world, so it does not seems resonable to me to have it.  

Of course the day an IPv6 DNSRBL will appear, we'll make the change to
support it.

Anyone else has an opinion on that point?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

milter-greylist 2.1.10

2006-07-31 by Emmanuel Dreyfus

Hello

Here is milter-greylist 2.1.10, which includes all the fixes that have been 
sent during the week-end.

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.10.tgz
MD5 (milter-greylist-2.1.10.tgz) = a987639aa56bce8cea7e27c5f772e426

Please test and report any remaining issues.

-- 
Emmanuel Dreyfus
manu@...

URL filtering?

2006-07-31 by Emmanuel Dreyfus

Hi

This is not related to milter-greylist, but perhaps you guys have some
expericence on the topic:

My domain is under a flood of spams for viagra. All the messages are 
different. The only thing they have in common is that the URL they
contain resolve to the same IP address (the domain name of the URL is
also different on each spam I saw). The IP is 211.144.68.87, FWIW.

In order to filter that, I should grab the URL from the message, 
resolve the address and blacklist if it's the infamous IP. 

That's not easy to do. Anyone knows a tool that already do that kind
of thing? 

I thought I could use a shell script hooked to a honeypot address to
collect all the domains resolving to this IP, and use milter-regex to
reject the messages containing the URL containing them, but it still
looks rather unefficient.

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] URL filtering?

2006-07-31 by Sysadmin

Tere.
> I thought I could use a shell script hooked to a honeypot address to
> collect all the domains resolving to this IP, and use milter-regex to
> reject the messages containing the URL containing them, but it still
> looks rather unefficient.
>
>   
Don't You use some other anti-spam tool? I don't, as I haven't found any 
good and fast and nonperl based software. Dspam is close, but I haven't 
yet figured out how to set it up with global quarantine.

Now, the sendmail with rbl -s + milter-greylist works pretty fine for 
me, however I'm also interested some more protection, about Your idea, 
maybe some procmail filter which parses the url and adds this to 
milter-greylist -s blacklist (we do need something like this) also will 
work. But this isn't actually the solution, as You told, all messages 
are different, so You catch the ip addresses which never return anyway.

-- 
Sysadmin

Re: [milter-greylist] URL filtering?

2006-07-31 by Emmanuel Dreyfus

On Mon, Jul 31, 2006 at 11:54:21AM +0300, Sysadmin wrote:
> Now, the sendmail with rbl -s + milter-greylist works pretty fine for 
> me, however I'm also interested some more protection, about Your idea, 
> maybe some procmail filter which parses the url and adds this to 
> milter-greylist -s blacklist (we do need something like this) also will 
> work. But this isn't actually the solution, as You told, all messages 
> are different, so You catch the ip addresses which never return anyway.

The URL in the message is always different but always resolves to the 
same IP. That's a good basis for filtering...

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] milter-greylist 2.1.10

2006-07-31 by Hajimu UMEMOTO

Hi,


>>>>> On Mon, 31 Jul 2006 08:28:55 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> Here is milter-greylist 2.1.10, which includes all the fixes that have been 
manu> sent during the week-end.

manu> http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.10.tgz
manu> MD5 (milter-greylist-2.1.10.tgz) = a987639aa56bce8cea7e27c5f772e426

manu> Please test and report any remaining issues.

Could you enclose bzero with HAVE_RESN?

Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Mon Jul 31 16:33:54 2006
+++ dnsrbl.c	Mon Jul 31 18:03:27 2006
@@ -114,7 +114,9 @@ dnsrbl_check_source(sa, salen, source)
 
 	blacklisted = SA(&source->d_blacklisted);
 
+#ifdef HAVE_RESN
 	bzero(&res, sizeof(res));
+#endif
 	res_ninit(&res);
 
 	reverse_endian(SA(&ss), sa);


My patch to query T_AAAA when blacklisted is AF_INET6 was not applied.
It is for consistency with other part, and it should be still applied
even when we decide not to support IPv6 DNSRBL.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] URL filtering?

2006-07-31 by Sysadmin

Tere.
>
> The URL in the message is always different but always resolves to the 
> same IP. That's a good basis for filtering...
>
>   
First You have to receive and parse message to find, does it include 
this url and later You can blacklist the sender's ip address, which is 
is always different.

Doing this, doesn't actually give You any relief, as in You cannot 
reject message in helo status.

-- 
Sysadmin

Re: [milter-greylist] URL filtering?

2006-07-31 by Emmanuel Dreyfus

On Mon, Jul 31, 2006 at 12:14:56PM +0300, Sysadmin wrote:
> > The URL in the message is always different but always resolves to the 
> > same IP. That's a good basis for filtering...
> >
> >   
> First You have to receive and parse message to find, does it include 
> this url and later You can blacklist the sender's ip address, which is 
> is always different.
> 
> Doing this, doesn't actually give You any relief, as in You cannot 
> reject message in helo status.

No, the idea is to reject messages containing (in the message body) the URL
with domains resolving to this unique IP.

-- 
Emmanuel Dreyfus
manu@...

Re: URL filtering?

2006-07-31 by Ugo Bellavance

--- In milter-greylist@yahoogroups.com, Emmanuel Dreyfus <manu@...> wrote:
>
> Hi
> 
> This is not related to milter-greylist, but perhaps you guys have some
> expericence on the topic:
> 
> My domain is under a flood of spams for viagra. All the messages are 
> different. The only thing they have in common is that the URL they
> contain resolve to the same IP address (the domain name of the URL is
> also different on each spam I saw). The IP is 211.144.68.87, FWIW.
> 
> In order to filter that, I should grab the URL from the message, 
> resolve the address and blacklist if it's the infamous IP. 
> 
> That's not easy to do. Anyone knows a tool that already do that kind
> of thing? 
> 

Hi,

http://www.snertsoft.com/sendmail/milter-link/

may help you.

Or spamassassin, which  uses SURBLs.

Ugo
Show quoted textHide quoted text
> I thought I could use a shell script hooked to a honeypot address to
> collect all the domains resolving to this IP, and use milter-regex to
> reject the messages containing the URL containing them, but it still
> looks rather unefficient.
> 
> -- 
> Emmanuel Dreyfus
> manu@...
>

Re: [milter-greylist] milter-greylist 2.1.10

2006-07-31 by Emmanuel Dreyfus

On Mon, Jul 31, 2006 at 06:12:55PM +0900, Hajimu UMEMOTO wrote:
> My patch to query T_AAAA when blacklisted is AF_INET6 was not applied.
> It is for consistency with other part, and it should be still applied
> even when we decide not to support IPv6 DNSRBL.
Could you please repost it? I can't find it

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] milter-greylist 2.1.10

2006-07-31 by Hajimu UMEMOTO

Hi,

>>>>> On Mon, 31 Jul 2006 11:56:56 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Mon, Jul 31, 2006 at 06:12:55PM +0900, Hajimu UMEMOTO wrote:
> My patch to query T_AAAA when blacklisted is AF_INET6 was not applied.
> It is for consistency with other part, and it should be still applied
> even when we decide not to support IPv6 DNSRBL.
manu> Could you please repost it? I can't find it

Okay, here it is.

Index: dnsrbl.c
diff -u -p dnsrbl.c.orig dnsrbl.c
--- dnsrbl.c.orig	Sat Jul 29 00:41:51 2006
+++ dnsrbl.c	Sat Jul 29 03:42:14 2006
@@ -103,10 +103,12 @@ dnsrbl_check_source(sa, salen, source)
 	int anslen;
 	ns_msg handle;
 	ns_rr rr;
-	int i;
+	int qtype, i;
 	char *dnsrbl = source->d_domain;
 	struct sockaddr *blacklisted;
 	int retval = 0;
+	char *addr;
+	size_t len;
 
 	blacklisted = SA(&source->d_blacklisted);
 
@@ -123,7 +125,27 @@ dnsrbl_check_source(sa, salen, source)
 	(void)strncat(req, ".", NS_MAXDNAME);
 	(void)strncat(req, dnsrbl, NS_MAXDNAME);
 
-	anslen = res_nquery(&res, req, C_IN, T_A, ans, sizeof(ans));
+	switch (blacklisted->sa_family) {
+	case AF_INET:
+		qtype = T_A;
+		addr = (char *)SADDR4(blacklisted);
+		len = sizeof(*SADDR4(blacklisted));
+		break;
+#ifdef AF_INET6
+	case AF_INET6:
+		qtype = T_AAAA;
+		addr = (char *)SADDR6(blacklisted);
+		len = sizeof(*SADDR6(blacklisted));
+		break;
+#endif
+	default:
+		syslog(LOG_ERR, "unexpected address family %d",
+		    blacklisted->sa_family);
+		exit(EX_SOFTWARE);
+		break;
+	}
+
+	anslen = res_nquery(&res, req, C_IN, qtype, ans, sizeof(ans));
 	if (anslen == -1)
 		goto end;
 
@@ -134,9 +156,6 @@ dnsrbl_check_source(sa, salen, source)
 	}
 
 	for (i = 0; i < ns_msg_count(handle, ns_s_an); i++) {
-		char *addr;
-		size_t len;
-
 		if ((ns_parserr(&handle, ns_s_an, i, &rr)) != 0) {
 			syslog(LOG_ERR, "ns_parserr failed: %s", 
 			    strerror(errno));
@@ -145,35 +164,16 @@ dnsrbl_check_source(sa, salen, source)
 		}
 
 		switch (blacklisted->sa_family) {
-		case AF_INET: {
-			struct sockaddr_in *sin;
-
+		case AF_INET:
 			if (rr.type != T_A)
 				continue;
-
-			sin = SA4(blacklisted);
-			addr = (char *)&sin->sin_addr;
-			len = sizeof(sin->sin_addr);
 			break;
-		}
 #ifdef AF_INET6
-		case AF_INET6: {
-			struct sockaddr_in6 *sin6;
-
+		case AF_INET6:
 			if (rr.type != T_AAAA)
 				continue;
-
-			sin6 = SA6(blacklisted);
-			addr = (char *)&sin6->sin6_addr;
-			len = sizeof(sin6->sin6_addr);
 			break;
-		}
 #endif
-		default:
-			syslog(LOG_ERR, "unexpected address family %d", 
-			    blacklisted->sa_family);
-			exit(EX_SOFTWARE);
-			break;
 		}
 
 		if (memcmp(addr, rr.rdata, len) == 0) {


Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Re: [milter-greylist] URL filtering?

2006-07-31 by Oliver Fromme

Emmanuel Dreyfus wrote:
 > This is not related to milter-greylist, but perhaps you guys have some
 > expericence on the topic:
 > 
 > My domain is under a flood of spams for viagra. All the messages are 
 > different. The only thing they have in common is that the URL they
 > contain resolve to the same IP address (the domain name of the URL is
 > also different on each spam I saw). The IP is 211.144.68.87, FWIW.
 > 
 > In order to filter that, I should grab the URL from the message, 
 > resolve the address and blacklist if it's the infamous IP. 

I don't know if this is helpful for you, but maybe it is.
Some time ago had a similar problem and wrote a small
script in Python.  It reads a single mail from stdin,
scans it for HTTP links, extracts the domain name, tries
to resolve the IP address and compares it with a hardcoded
list of addresses.  It exits with code 1 if a match is
found, otherwise exit code 0.  I call it using maildrop
(see http://www.courier-mta.org/maildrop/ ) from my user's
~/.mailfilter file, but you can also use it with procmail
or any other local delivery agent.  Maybe it can also
called from the MTA, but I haven't tried that.

Here's the script:

    #!/usr/bin/env python
    
    import re, socket, sys
    
    ip_blacklist = (      # These are just examples!
            "1.2.3.4", "33.55.77.99", "100.50.200.5",
            "221.221.147.254", "211.233.178.123"
    )
    
    url_regexpr = r"http://([-.a-z0-9]+\.[a-z]{2,})"
    
    verbose = False
    
    regex = re.compile(url_regexpr, re.I | re.M)
    black = dict(zip(ip_blacklist, range(len(ip_blacklist))))
    
    for match in regex.finditer(sys.stdin.read()):
            name = match.group(1)
            try:
                    ip = socket.gethostbyname(name)
                    if verbose:
                            print name, "-->", ip
                    if ip in black:
                            sys.exit (1)
            except socket.gaierror:
                    if verbose:
                            print name, "DOES NOT RESOLVE"
    
    sys.exit (0)

It could certainly be improved, e.g. by reading the list
of IP addresses from a file, or by reading more than just
one mail message.  Also note that it only handles plain
test mails, it doesn't handle quoted-printable, base64
encodings etc.  It also depends on the local nameserver's
caching, so it doesn't do its own caching (which could
easily be implemented).

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"That's what I love about GUIs: They make simple tasks easier,
and complex tasks impossible."
        -- John William Chambless

milter-greylist 2.1.12

2006-08-01 by manu@netbsd.org

Denis Solovyov <elk@...> wrote:

> Blacklisting  is  much  more  serious thing than greylisting, so I think
> there should be a directive to set a user-defined "go away" message with
> a  specific  explanation  on  why  one is blacklisted. As for me I would
> appreciate at least one global "go away" custom message.  Example:
> 
> goawaymsg "You are blacklisted by our antispam software - please see
> http://some.url/antispam-policy"

You have it! 2.1.12 can do that:

acl blacklist dnsrbl "heavyspam" msg "you are caught by heavyspam"

SMTP code and extended code can also be specified. And it also works for
acl greylist entries. Check the man page for details.

MD5 (milter-greylist-2.1.12.tgz) = 8582b573a90b2092654cc34fbe8ab6ee
http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.1.12.tgz

I also integrated IPv6 fixes from Hajimu UMEMOTO and a FreeBSD build fix
found by Bill Levering.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.12

2006-08-02 by Denis Solovyov

>> goawaymsg "You are blacklisted by our antispam software - please see
>> http://some.url/antispam-policy"
mno> You have it! 2.1.12 can do that:
mno> acl blacklist dnsrbl "heavyspam" msg "you are caught by heavyspam"

Only  per-acl messages?  Is there a possibility to make a custom default
(i.e. user-defined default message for those acls with undefined "msg")?

Best regards,
Denis Solovyov

Re: [milter-greylist] milter-greylist 2.1.12

2006-08-02 by manu@netbsd.org

Denis Solovyov <elk@...> wrote:

> Only  per-acl messages?  Is there a possibility to make a custom default
> (i.e. user-defined default message for those acls with undefined "msg")?

Patch the sources.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist 2.1.12

2006-08-02 by Emmanuel Dreyfus

On Wed, Aug 02, 2006 at 01:19:02PM +0200, manu@... wrote:
> > Only  per-acl messages?  Is there a possibility to make a custom default
> > (i.e. user-defined default message for those acls with undefined "msg")?
> 
> Patch the sources.

Addendum: Patch if you need it. Here is a sample setup, I'm confident you'll 
find fome ideas to fullfill your goals.

dnsrbl "bl1" bl1.example.net 127.0.0.1
dnsrbl "bl2" bl2.example.net 127.0.0.1

list "spamtraps" rcpt { \
	spamtrap1@... \
	spamtrap2@... \
	spamtrap3@... \
}
list "dnsrbl" dnsrbl { "bl1" "bl2" }

acl blacklist list "spamtraps" msg "you were caught in our spamtraps"
acl blacklist list "dnsrbl" msg "you were caught in a DNSRBL"
acl greylist default msg "see you later"


-- 
Emmanuel Dreyfus
manu@...

[Fwd: watch-greylist]

2006-08-02 by Chris Hoogendyk

We are running milter-greylist-1.6 on Solaris 9 with Sendmail 8.13.6,
etc. on an E250 with dual sparc processors.

We didn't go to milter-greylist-2.0.2 when building the system, because
we experienced what appeared to be memory leaks when trying to run it.
These seem to be in 1.6, but seemed to be worse in 2.0.2.

Periodically, with 1.6 running, we have a situation where the floodgates
open and we start getting spam. On checking mail logs and processes, we
discover that milter-greylist has stopped running. Because of this, I
have a cron script that checks this condition and restarts
milter-greylist. It doesn't trip off very often (a couple of times in a
few months). However, when it does, we find that milter-greylist has not
been functioning for several hours, but only actually terminates after
it has been this way for several hours. In the log files, we find
entries like:

     Jul 30 03:10:45 marlin sendmail[10724]: [ID 801593 mail.info]
k6U7AjxJ010724: Milter (greylist): to error state

Couple of questions:

Why does it do this? Are there many people testing on Solaris? When you
get 2.1.x ironed out, can we be a little more sure that stuff like this
has been cleaned up?

If it can see that it has gone "to error state", why shouldn't it just
terminate on the spot? Does it think it will come out of error state?
How about having an ability to restart itself?


---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4
Show quoted textHide quoted text
-------- Original Message --------
Subject: 	watch-greylist
Date: 	Wed, 2 Aug 2006 11:55:10 -0400
From: 	George Drake <george@...>
To: 	hoogendyk@...



Chris, given that the log contains lines like
Jul 30 03:10:45 marlin sendmail[10724]: [ID 801593 mail.info] 
k6U7AjxJ010724: Milter (greylist): to
error state

I think I'll scan the log file looking for the following at the end 
of a line.  That should make the string search fast since I can lock 
it to the end of a line.

Milter (greylist): to error state

Are you aware of any other cases in which this string might occur and 
not signify we should kill and restart milter greylist?
-- 
--------
	George Drake		413-545-2625, FAX 413-545-3243
	Dept. of Biology, Univ. of Mass., Amherst, MA 01003
	  Director of Computing and Networking
	  College of Natural Sciences and Mathematics
	  University of Massachusetts, Amherst

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by Matthias Scheler

On Wed, Aug 02, 2006 at 12:23:10PM -0400, Chris Hoogendyk wrote:
> We are running milter-greylist-1.6 on Solaris 9 with Sendmail 8.13.6,
> etc. on an E250 with dual sparc processors.
> 
> We didn't go to milter-greylist-2.0.2 when building the system, because
> we experienced what appeared to be memory leaks when trying to run it.
> These seem to be in 1.6, but seemed to be worse in 2.0.2.

Are you sure this is not a Solaris bug? Quoting from an e-mail which
somebody sent to this list a while ago:

	Ok, I found the problem. Milter-greylist is innocent. It's a Solaris 
	bug. Solaris 8 needs the 108827-40 patch (which has since then been 
	obsoleted by 108993-39). There seem to be severe leaks in
	getaddrinfo() and friends in non-patched libpthread/libsocket/libnsl.

Solaris 9 might require a similar patch.

> If it can see that it has gone "to error state", why shouldn't it just
> terminate on the spot? Does it think it will come out of error state?

The "error state" message is printed out by "sendmail", not by
"milter-greylist".

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by Oliver Fromme

Matthias Scheler wrote:
 > Chris Hoogendyk wrote:
 > > We are running milter-greylist-1.6 on Solaris 9 with Sendmail 8.13.6,
 > > etc. on an E250 with dual sparc processors.
 > > 
 > > We didn't go to milter-greylist-2.0.2 when building the system, because
 > > we experienced what appeared to be memory leaks when trying to run it.
 > > These seem to be in 1.6, but seemed to be worse in 2.0.2.
 > 
 > Are you sure this is not a Solaris bug? Quoting from an e-mail which
 > somebody sent to this list a while ago:
 > 
 > 	Ok, I found the problem. Milter-greylist is innocent. It's a Solaris 
 > 	bug. Solaris 8 needs the 108827-40 patch (which has since then been 
 > 	obsoleted by 108993-39). There seem to be severe leaks in
 > 	getaddrinfo() and friends in non-patched libpthread/libsocket/libnsl.
 > 
 > Solaris 9 might require a similar patch.

A friend of mine is running milter-greylist 2.1.3 on
Solaris 9 without problems.  The process size grows at
the beginning (which is normal because the greylist
grows).  The growth slows down eventually (slowly).
NB:  "uname" information from that machine, if it helps:
SunOS 5.9 Generic_117171-02 sun4u sparc SUNW,UltraAX-i2

You should make sure that you are up-to-date with Sun's
recommended patches, no matter what version of Solaris
you're running.

I noticed that the memory handling (malloc()/free()) of
milter-greylist is a pretty tough test for the system's
malloc implementation.  Solaris, Linux and the BSDs all
have very different implementations.  Some of them are
optimized for speed, some for low memory consumption,
and some try be good at both.  So if you think you see
a memory leak, it might also be an artefact of your
system's malloc implementation.  If your system's imple-
mentation can be configured (such as FreeBSD's, see
the malloc(3) manual page), you could give it a try to
tune it better for milter-greylist.

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"I made up the term 'object-oriented', and I can tell you
I didn't have C++ in mind."
        -- Alan Kay, OOPSLA '97

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by Chris Hoogendyk

Matthias Scheler wrote:
> On Wed, Aug 02, 2006 at 12:23:10PM -0400, Chris Hoogendyk wrote:
>   
>> We are running milter-greylist-1.6 on Solaris 9 with Sendmail 8.13.6,
>> etc. on an E250 with dual sparc processors.
>>
>> We didn't go to milter-greylist-2.0.2 when building the system, because
>> we experienced what appeared to be memory leaks when trying to run it.
>> These seem to be in 1.6, but seemed to be worse in 2.0.2.
>>     
>
> Are you sure this is not a Solaris bug? Quoting from an e-mail which
> somebody sent to this list a while ago:
>
> 	Ok, I found the problem. Milter-greylist is innocent. It's a Solaris 
> 	bug. Solaris 8 needs the 108827-40 patch (which has since then been 
> 	obsoleted by 108993-39). There seem to be severe leaks in
> 	getaddrinfo() and friends in non-patched libpthread/libsocket/libnsl.
>
> Solaris 9 might require a similar patch.
>   

I'm not sure if it represents the same problem, but patch 112839 on
Solaris 9 applies to the same library components. It's current rev is 8.
Typically, Solaris 8 & 9 are rather different in a lot of ways, so I
wouldn't necessarily expect a bug in 8 on a component that had gone
through 40 patches to have the same history on 9. In any case, my system
has been patched.

If that were the problem, wouldn't I expect to see this cropping up in a
lot of other places? I'm running Sendmail, mimedefang, spamassassin,
milter-greylist, poprelayd, uw-imap, procmail, sasl-authd, samba, apache
2, php, squirrelmail, mysql, msql, berkeleydb, majordomo, hypermail,
tikiwiki, pro-ftp, netatalk, jetdirect, as well as all the development
tools. The server processes over 50,000 mail messages a day and has 24
drives hanging off it for research space, web space and data.

I don't want to put down milter-greylist, because it is a godsend for
reducing spam. But, it is the only process that I have to have watched
by a cron to make sure it stays up.

We have milter-greylist running on a couple of other systems. One is 
nearly identical, but doesn't serve as large a volume of mail. It
experiences failures of milter-greylist less frequently. The other is an
OpenBSD system on a PC that serves a small office. It has never
experienced failures. One aspect of this comparison is that the OpenBSD
system handles probably 1/100th of the mail volume. Another aspect is
that it is a single processor system. Both of the Sun servers are dual
processor. This comparison could indicate that the problem accumulates
over volume of processing, and/or that it is in some way not thread safe
on a multiprocessor system. But that's guessing.

>> If it can see that it has gone "to error state", why shouldn't it just
>> terminate on the spot? Does it think it will come out of error state?
>>     
>
> The "error state" message is printed out by "sendmail", not by
> "milter-greylist".
>
> 	Kind regards
>   

---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by Chris Hoogendyk

Oliver Fromme wrote:
> Matthias Scheler wrote:
>  > Chris Hoogendyk wrote:
>  > > We are running milter-greylist-1.6 on Solaris 9 with Sendmail 8.13.6,
>  > > etc. on an E250 with dual sparc processors.
>  > > 
>  > > We didn't go to milter-greylist-2.0.2 when building the system, because
>  > > we experienced what appeared to be memory leaks when trying to run it.
>  > > These seem to be in 1.6, but seemed to be worse in 2.0.2.
>  > 
>  > Are you sure this is not a Solaris bug? Quoting from an e-mail which
>  > somebody sent to this list a while ago:
>  > 
>  > 	Ok, I found the problem. Milter-greylist is innocent. It's a Solaris 
>  > 	bug. Solaris 8 needs the 108827-40 patch (which has since then been 
>  > 	obsoleted by 108993-39). There seem to be severe leaks in
>  > 	getaddrinfo() and friends in non-patched libpthread/libsocket/libnsl.
>  > 
>  > Solaris 9 might require a similar patch.
>
> A friend of mine is running milter-greylist 2.1.3 on
> Solaris 9 without problems.  The process size grows at
> the beginning (which is normal because the greylist
> grows).  The growth slows down eventually (slowly).
> NB:  "uname" information from that machine, if it helps:
> SunOS 5.9 Generic_117171-02 sun4u sparc SUNW,UltraAX-i2
>   

That's a SunFire V120, which has a single sparc processor. *If* we are
talking about an issue that is related to not being thread safe, then it
might not have a problem on that machine. Do you know what volume of
mail they handle?

We saw that growth also when we tried 2.0.2. It seemed extreme and
counter to our experience running with the 1.6 version. Does that make
sense? Is there a significant difference in the way the two versions are
handling things?

> You should make sure that you are up-to-date with Sun's
> recommended patches, no matter what version of Solaris
> you're running.
>   
yup.

> I noticed that the memory handling (malloc()/free()) of
> milter-greylist is a pretty tough test for the system's
> malloc implementation.  Solaris, Linux and the BSDs all
> have very different implementations.  Some of them are
> optimized for speed, some for low memory consumption,
> and some try be good at both.  So if you think you see
> a memory leak, it might also be an artefact of your
> system's malloc implementation.  If your system's imple-
> mentation can be configured (such as FreeBSD's, see
> the malloc(3) manual page), you could give it a try to
> tune it better for milter-greylist.
>   

From the Solaris 9 man pages:

USAGE
     Portable applications should avoid using valloc() but should
     instead  use  malloc()  or  mmap(2). On systems with a large
     page size, the  number  of  successful  valloc()  operations
     might be 0.

     Comparative features of malloc(3C), bsdmalloc(3MALLOC),  and
     malloc(3MALLOC) are as follows:

        o  The bsdmalloc(3MALLOC) routines afford better  perfor-
           mance, but are space-inefficient.

        o  The malloc(3MALLOC) routines are space-efficient,  but
           have slower performance.

        o  The standard, fully SCD-compliant malloc routines  are
           a trade-off between performance and space-efficiency.


---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by manu@netbsd.org

Oliver Fromme <olli@...> wrote:

> I noticed that the memory handling (malloc()/free()) of
> milter-greylist is a pretty tough test for the system's
> malloc implementation. 

Wow. I never expected the legitimate use of malloc() and free() to be a
torture test for the system's libc. I'm a bit shocked to hear that it
can cause troubes.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by manu@netbsd.org

Chris Hoogendyk <hoogendyk@...> wrote:

> That's a SunFire V120, which has a single sparc processor. *If* we are
> talking about an issue that is related to not being thread safe, then it
> might not have a problem on that machine. Do you know what volume of
> mail they handle?

milter-greylist uses several threads even on uniprocessor systems.
That's the way libmilter architecture was designed: each SMTP connexion
causes a thread creation. No way to work around this.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by manu@netbsd.org

Chris Hoogendyk <hoogendyk@...> wrote:

> We have milter-greylist running on a couple of other systems. One is 
> nearly identical, but doesn't serve as large a volume of mail. 

You are sure milter-greylist does not get killed by a system limit,
right?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by Chris Hoogendyk

manu@... wrote:
> Chris Hoogendyk <hoogendyk@...> wrote:
>
>   
>> We have milter-greylist running on a couple of other systems. One is 
>> nearly identical, but doesn't serve as large a volume of mail. 
>>     
>
> You are sure milter-greylist does not get killed by a system limit,
> right?
>   

I'm not sure what sort of limit you're talking about here.


---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-02 by manu@netbsd.org

Chris Hoogendyk <hoogendyk@...> wrote:

> >> We have milter-greylist running on a couple of other systems. One is
> >> nearly identical, but doesn't serve as large a volume of mail. 
> > You are sure milter-greylist does not get killed by a system limit,
> > right?
> I'm not sure what sort of limit you're talking about here.

The kernel-enforced limits to memory usage, settable by ulimit (or limit
for csh).

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-03 by Oliver Fromme

manu@... wrote:
 > Oliver Fromme wrote:
 > > I noticed that the memory handling (malloc()/free()) of
 > > milter-greylist is a pretty tough test for the system's
 > > malloc implementation. 
 > 
 > Wow. I never expected the legitimate use of malloc() and free() to be a
 > torture test for the system's libc. I'm a bit shocked to hear that it
 > can cause troubes.

It should not cause any trouble.  But the efficiency depends
on the respective malloc implementation.  Some of them are
optimized for speed, but can lead to "memory fragmentation"
if a programm uses a lot of malloc+free (or realloc) with
varying sizes (which milter-greylist does, AFAIK).  Others
are optimized for avoiding such fragmentation, but at the
cost of speed.

That means that milter-greylist can exhibit very different
memory behaviour on different platforms.  E.g. its memory
consumption might be moderate on one platform, while it
grows considerably larger on another platform, even if it's
under the same load and has the same number of entries in
its greylist.  Even on the same platform, the behavious can
be different.  As Chris Hoogendyk explained, Solaris seems
to offer three different malloc implementations.  FreeBSD
got an alternative one recently, and both of them are
configurable (globally via /etc/malloc.con, and per-process
via an environment variable).

Bottom line:  As a programmer, you should not need to care
about a system's malloc efficiency.  Just try to trust it.
Well, there are certain ways to assist malloc() doing a
more efficient job, e.g. by allocating larger chunks of
memory and subdividing them yourself, so you have to call
malloc or realloc less often.  But it's difficult to say
whether that's really worth the effort.

Best regards
   Oliver

PS:  There's no reason to be shocked.  :-)

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"Being really good at C++ is like being really good
at using rocks to sharpen sticks."
        -- Thant Tessman

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-03 by Matthias Scheler

On Wed, Aug 02, 2006 at 03:08:54PM -0400, Chris Hoogendyk wrote:
> I'm not sure if it represents the same problem, but patch 112839 on
> Solaris 9 applies to the same library components. It's current rev is 8.
> Typically, Solaris 8 & 9 are rather different in a lot of ways, so I
> wouldn't necessarily expect a bug in 8 on a component that had gone
> through 40 patches to have the same history on 9. In any case, my system
> has been patched.

As somebody who has worked in Sun's Solaris engineering department for
several years I can assure that a considerable number of bugs where
fixed in the Solaris 10 development sources (long before its release)
and the fixes were backported to Solaris 9 and 8 (or even further)
afterwards.

> If that were the problem, wouldn't I expect to see this cropping up in a
> lot of other places?

Not necessarily.

> I'm running Sendmail, mimedefang, spamassassin,
> milter-greylist, poprelayd, uw-imap, procmail, sasl-authd, samba, apache
> 2, php, squirrelmail, mysql, msql, berkeleydb, majordomo, hypermail,
> tikiwiki, pro-ftp, netatalk, jetdirect, as well as all the development
> tools. The server processes over 50,000 mail messages a day and has 24
> drives hanging off it for research space, web space and data.

A lot of these program ...
1.) ... don't use getaddrinfo() at all but older functions like
    gethostbyname().
2.) ... don't perform many DNS queries.
3.) ... create child process to deal with incoming connections. And when
    the child process terminates all the leaked memory is freed.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-07 by Chris Hoogendyk

manu@... wrote:
> Chris Hoogendyk <hoogendyk@...> wrote:
>
>   
>>>> We have milter-greylist running on a couple of other systems. One is
>>>> nearly identical, but doesn't serve as large a volume of mail. 
>>>>         
>>> You are sure milter-greylist does not get killed by a system limit,
>>> right?
>>>       
>> I'm not sure what sort of limit you're talking about here.
>>     
>
> The kernel-enforced limits to memory usage, settable by ulimit (or limit
> for csh).
>   

so, would stack or number of files seem to be a problem from 
milter-greylist's point of view?

# ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 8192
coredump(blocks) unlimited
nofiles(descriptors) 256
memory(kbytes) unlimited
#


I got another failure this morning.

The only entry in /var/adm/messages around the failure was:

Aug  7 07:31:22 bleep genunix: [ID 603404 kern.notice] NOTICE: core_log: 
milter-greylist[19992] setid process, core not dumped: 
/var/core/core.milter-greylist.19992



The surrounding entries in /var/adm/mail.log were:

Aug  7 07:31:19 bleep milter-greylist: [ID 596769 mail.debug] Looking up 
59.162.254.64 in /etc/mail/popip.db
Aug  7 07:31:19 bleep milter-greylist: [ID 575326 mail.debug] 
59.162.254.64 was not in database
Aug  7 07:31:19 bleep milter-greylist: [ID 596769 mail.debug] Looking up 
59.162.254.64 in /etc/mail/popip.db
Aug  7 07:31:19 bleep milter-greylist: [ID 575326 mail.debug] 
59.162.254.64 was not in database
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVDVf020107: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep1@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVEIU020109: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep2@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVDp8020105: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep3@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVET4020108: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep4@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVD3g020106: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep5@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVEv2020110: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep6@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 226885 mail.debug] 
k77BVE8N020111: addr = 59.162.254.64, from = <aadv11422@...>, 
rcpt = <bleep7@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep1@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 829939 mail.info] 
k77BVDVf020107: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep1@...>: autowhitelisted for 20:00:00
Aug  7 07:31:21 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep4@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 829939 mail.info] 
k77BVET4020108: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep4@...>: autowhitelisted for 20:00:00
Aug  7 07:31:21 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep7@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 829939 mail.info] 
k77BVE8N020111: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep7@...>: autowhitelisted for 20:00:00
Aug  7 07:31:21 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep6@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 829939 mail.info] 
k77BVEv2020110: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep6@...>: autowhitelisted for 20:00:00
Aug  7 07:31:21 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep5@...>
Aug  7 07:31:21 bleep milter-greylist: [ID 829939 mail.info] 
k77BVD3g020106: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep5@...>: autowhitelisted for 20:00:00
Aug  7 07:31:22 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep2@...>
Aug  7 07:31:22 bleep milter-greylist: [ID 829939 mail.info] 
k77BVEIU020109: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep2@...>: autowhitelisted for 20:00:00
Aug  7 07:31:22 bleep milter-greylist: [ID 333256 mail.debug] removed: 
59.162.254.64 from <aadv11422@...> to <bleep3@...>
Aug  7 07:31:22 bleep milter-greylist: [ID 829939 mail.info] 
k77BVDp8020105: addr 59.162.254.64 from <aadv11422@...> rcpt 
<bleep3@...>: autowhitelisted for 20:00:00
Aug  7 07:31:22 bleep milter-greylist: [ID 596769 mail.debug] Looking up 
59.162.254.64 in /etc/mail/popip.db
Aug  7 07:31:22 bleep milter-greylist: [ID 575326 mail.debug] 
59.162.254.64 was not in database
Aug  7 07:31:22 bleep milter-greylist: [ID 596769 mail.debug] Looking up 
59.162.254.64 in /etc/mail/popip.db
Aug  7 07:31:22 bleep last message repeated 1 time
Aug  7 07:31:22 bleep milter-greylist: [ID 575326 mail.debug] 
59.162.254.64 was not in database
Aug  7 07:31:22 bleep sendmail[20114]: [ID 801593 mail.error] 
k77BVHlG020114: milter_read(greylist): cmd read returned 0, expecting 5
Aug  7 07:31:22 bleep sendmail[20114]: [ID 801593 mail.info] 
k77BVHlG020114: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20115]: [ID 801593 mail.error] 
k77BVHs9020115: Milter (greylist): write(D) returned -1, expected 23: 
Broken pipe
Aug  7 07:31:22 bleep sendmail[20115]: [ID 801593 mail.info] 
k77BVHs9020115: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20116]: [ID 801593 mail.error] 
k77BVHLA020116: milter_read(greylist): cmd read returned 0, expecting 5
Aug  7 07:31:22 bleep sendmail[20116]: [ID 801593 mail.info] 
k77BVHLA020116: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20119]: [ID 801593 mail.error] 
k77BVL3Y020119: Milter (greylist): write(D) returned -1, expected 6: 
Broken pipe
Aug  7 07:31:22 bleep sendmail[20119]: [ID 801593 mail.info] 
k77BVL3Y020119: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20118]: [ID 801593 mail.error] 
k77BVLpK020118: Milter (greylist): write(D) returned -1, expected 6: 
Broken pipe
Aug  7 07:31:22 bleep sendmail[20118]: [ID 801593 mail.info] 
k77BVLpK020118: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20120]: [ID 801593 mail.error] 
k77BVMwY020120: Milter (greylist): error connecting to filter: 
Connection refused by /var/milter-greylist/greylist.sock
Aug  7 07:31:22 bleep sendmail[20120]: [ID 801593 mail.info] 
k77BVMwY020120: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20121]: [ID 801593 mail.error] 
k77BVMDQ020121: Milter (greylist): error connecting to filter: 
Connection refused by /var/milter-greylist/greylist.sock
Aug  7 07:31:22 bleep sendmail[20121]: [ID 801593 mail.info] 
k77BVMDQ020121: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20122]: [ID 801593 mail.error] 
k77BVMHM020122: Milter (greylist): error connecting to filter: 
Connection refused by /var/milter-greylist/greylist.sock
Aug  7 07:31:22 bleep sendmail[20122]: [ID 801593 mail.info] 
k77BVMHM020122: Milter (greylist): to error state
Aug  7 07:31:22 bleep sendmail[20113]: [ID 801593 mail.error] 
k77BVHq8020113: Milter (greylist): write(D) returned -1, expected 59: 
Broken pipe
Aug  7 07:31:22 bleep sendmail[20113]: [ID 801593 mail.info] 
k77BVHq8020113: Milter (greylist): to error state



TIA



---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-07 by manu@netbsd.org

Chris Hoogendyk <hoogendyk@...> wrote:

> so, would stack or number of files seem to be a problem from 
> milter-greylist's point of view?

You can always unlimit them to check if there is a change.

Could you run milter-greylist within gdb and collect a backtrace at
crash time? That would help

# gdb milter-greylist
(gdb) run -Dv

when it crashes, run the bt command.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-07 by Chris Hoogendyk

manu@... wrote:
> Chris Hoogendyk <hoogendyk@...> wrote:
>
>   
>> so, would stack or number of files seem to be a problem from 
>> milter-greylist's point of view?
>>     
>
> You can always unlimit them to check if there is a change.
>
> Could you run milter-greylist within gdb and collect a backtrace at
> crash time? That would help
>
> # gdb milter-greylist
> (gdb) run -Dv
>
> when it crashes, run the bt command.
>   

ok, so I have two options here:

one is aimed at possibly solving my problem directly without answering 
why it's happening. That would be to add `ulimit -n unlimit` to the 
/etc/init.d/greylist startup script. Maybe it would solve the problem. 
Maybe it wouldn't.

the other option is to run milter-greylist from within the gnu debugger 
and wait for it to crash. this would contribute more in terms of 
figuring out why it's happening.

Please note that this is a departmental production server handling a 
large volume of mail. If I'm playing around and cause milter-greylist 
not to function for the better part of a day, there will be hundreds of 
users pissed at me for letting all that spam come flying through. So, 
with that in mind, . . . .

If I'm to run milter-greylist under gdb, I will have to run it from a 
terminal session. Fortunately, my desktop machine is on an APC 
Smart-UPS, and it is BSD based. So, typically, my desktop machine is up 
and running for months at a time with terminal sessions opened from one 
patch rev to the next (that typically being when I have to reboot). 
However, this means that my machine becomes semi-critical rather than 
just my tool. I may have to wait a week or a month for a crash of 
milter-greylist. Fortunately, again, I do have the fallback cron script 
on the server that will re-start milter-greylist if/when it crashes. But 
then, it will be running without a terminal session or a debugger. So, . 
. . .

I open a terminal session to the server, source the first half of the 
/etc/init.d/greylist to get the environment variables set:

     #! /bin/sh

     # Greylist init script
     # July 2004
     # BERTRAND Joel

     LD_LIBRARY_PATH=/usr/local/BerkeleyDB.4.4/lib;
     export LD_LIBRARY_PATH;

     # Based on skeleton by Miquel van Smoorenburg and Ian Murdock

     PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
     DAEMON=/usr/local/bin/milter-greylist
     NAME=milter-greylist
     SNAME=greylist
     DESC="Greylist Mail Filter Daemon"
     PIDFILE="/var/run/$NAME.pid"
     PNAME="milter-greylist"
     USER="smmsp"
     #USER="root"
     CONFILE=/etc/mail/greylist.conf
     #SOCKET=/var/run/milter-greylist/greylist.sock
     SOCKET=/var/milter-greylist/greylist.sock
     POPIPDB=/etc/mail/popip.db
     # This is standard...
     DOPTIONS="-b $POPIPDB -P $PIDFILE -u $USER -p $SOCKET"
     # This is for debug...
     #DOPTIONS="-v -b $POPIPDB -P $PIDFILE -u $USER -p $SOCKET"

     TMPDIR=/tmp;
     export TMPDIR;
     # Apparently people have trouble if this isn't explicitly set...


and then

# dbg $DAEMON $DOPTIONS
(dbg) run -Dv

wait until I get a crash, and then

(dbg) backtrace

and send you the output


I'm assuming that if it crashes during the night, and the cron script 
restarts it on the server, my state within the debugger will remain 
unaltered and I will be able to do the backtrace.

And, while that is running, it will behave just like milter-greylist 
running off the init scripts free of any terminal sessions.

If that last sentence is true, then I can try this.



---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-07 by Emmanuel Dreyfus

On Mon, Aug 07, 2006 at 05:24:35PM -0400, Chris Hoogendyk wrote:
> If I'm to run milter-greylist under gdb, I will have to run it from a 
> terminal session. Fortunately, my desktop machine is on an APC 
> Smart-UPS, and it is BSD based. So, typically, my desktop machine is up 
> and running for months at a time with terminal sessions opened from one 
> patch rev to the next (that typically being when I have to reboot). 
> However, this means that my machine becomes semi-critical rather than 
> just my tool. I may have to wait a week or a month for a crash of 
> milter-greylist. 

Run the session within screen. That way your machine can go offline 
without interrupting the session.

> And, while that is running, it will behave just like milter-greylist 
> running off the init scripts free of any terminal sessions.

That's what the programmer wanted to implement, at least.

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] [Fwd: watch-greylist]

2006-08-08 by Oliver Fromme

Chris Hoogendyk wrote:
 > manu@... wrote:
 > > Chris Hoogendyk <hoogendyk@...> wrote:
 > > > so, would stack or number of files seem to be a problem from 
 > > > milter-greylist's point of view?
 > > 
 > > You can always unlimit them to check if there is a change.
 > > [...]
 > 
 > ok, so I have two options here:
 > 
 > one is aimed at possibly solving my problem directly without answering 
 > why it's happening. That would be to add `ulimit -n unlimit` to the 
 > /etc/init.d/greylist startup script. Maybe it would solve the problem. 
 > Maybe it wouldn't.

I don't think the stack limit is a problem for milter-
greylist.  But the number of file descriptors (which is
limited to 256 on your system, IIRC) may well be a
problem, especially if your mail server is quite busy.

Therefore lifting the limit of the number of file
descriptors to at least 1024 is certainly worth a try.

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"I invented Ctrl-Alt-Delete, but Bill Gates made it famous."
        -- David Bradley, original IBM PC design team

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.