Milter-greylist

I am running milter-greylist on the following linux system:

Kernel: 2.4.4 SMP (2 processors)
GLIBC:  2.2.2
             total       used       free     shared    buffers     cached
Mem:       5922468    4568268    1354200          0     600288    3237844
-/+ buffers/cache:     730136    5192332
Swap:      2097136    2097136          0

# cat /proc/sys/fs/file-max 
32768
# cat /proc/sys/kernel/threads-max 
786432

Without the sync feature enabled, the milter runs very well.  But if I
enable syncing between my 3 MX servers, then they all crash within an
hour or two.  We receive about 80000 emails per day.

I've compiled the bind 8 libraries with threads, and linked
milter-greylist to those (even though I am not using SPF it was still
linking to libresolv, so I modified it to use libdns):

        libdns.so.11 => /usr/local/lib/libdns.so.11 (0x4001e000)
        libnsl.so.1 => /lib/libnsl.so.1 (0x4010b000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x40122000)
        libc.so.6 => /lib/libc.so.6 (0x40138000)
        libisc.so.7 => /usr/local/lib/libisc.so.7 (0x40255000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

With syncing enabled, I get these error messages:

   Oct 13 11:32:50 XXX milter-greylist: greylist: thread_create()
   failed: 11, try again

which confuses me, because I can't find "thread_create" in the
milter-greylist source.  I assumed it was coming from libresolv but
I have linked libbind instead and the error still occurs.

Eventually I start seeing these errors:

   Oct 13 11:30:30 XXX milter-greylist: incoming connexion from
   XXX.XXX.XXX.XXX failed, In sync.c:692 pthread_create failed: Resource
   temporarily unavailable

The __LINE__ macro and __FILE__ macros were used to determine the
cause.  This is the ONLY line where I get the failure.  After a few
dozen of these milter-greylist completely dies.

Our resident C expert thinks I am hitting some kind of max threads per
user limit or max open files limit (I am running the milter under a
non-privileged user).  So I wrote a little shell script that counts
processes with 'ps -ef' and open files with 'lsof -i' and both never
get above 400 and 300, respectively.  [Unfortunately my distro's
version of ps does not support the -m flag for ps.]

Does anyone have any suggestions to resolv this, or has anyone
experienced similar problems when using sync?


Thanks,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Thu, Oct 13, 2005 at 02:29:02PM -0400, Matthew S. Cramer wrote:
>    Oct 13 11:32:50 XXX milter-greylist: greylist: thread_create()
>    failed: 11, try again
> 
> which confuses me, because I can't find "thread_create" in the
> milter-greylist source.  I assumed it was coming from libresolv but
> I have linked libbind instead and the error still occurs.

libmilter does the threading job, so thread_create is in libmilter

> Our resident C expert thinks I am hitting some kind of max threads per
> user limit or max open files limit (I am running the milter under a
> non-privileged user). 

That's also my feeling. Do you know how many thread you can run?

-- 
Emmanuel Dreyfus
manu@...

On Thu, Oct 13, 2005 at 07:41:03PM +0000, Emmanuel Dreyfus wrote:
> On Thu, Oct 13, 2005 at 02:29:02PM -0400, Matthew S. Cramer wrote:

> > Our resident C expert thinks I am hitting some kind of max threads per
> > user limit or max open files limit (I am running the milter under a
> > non-privileged user). 
> 
> That's also my feeling. Do you know how many thread you can run?

I am going to brush off my C skills and write a short program that
just spawns threads and counts them to see how long it can go before
it blows up.

Are the number of threads for syncing a factor of - sendmail children,
or number of tuples that need to be synced?

I'll write that program and let you know the results.

Actually - is there a global structure to which I can add a thread
count and then increment during any pthread_create (and decrement
during the destroy)?

BTW, I am using 2.0.1 of milter-greylist and sendmail 8.13.4.


Thanks,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Thu, Oct 13, 2005 at 04:53:00PM -0400, Matthew S. Cramer wrote:
> Are the number of threads for syncing a factor of - sendmail children,
> or number of tuples that need to be synced?

You have one thread per mail being processed and one thread for syncing.

> I'll write that program and let you know the results.
> 
> Actually - is there a global structure to which I can add a thread
> count and then increment during any pthread_create (and decrement
> during the destroy)?

You can add a global and change it in mlfi_connect (+1) and mlfi_close (-1)

-- 
Emmanuel Dreyfus
manu@...

On Thu, 2005-10-13 at 14:29 -0400, Matthew S. Cramer wrote:
> Kernel: 2.4.4 SMP (2 processors)
> GLIBC:  2.2.2
>              total       used       free     shared    buffers
> cached
> Mem:       5922468    4568268    1354200          0     600288
> 3237844
> -/+ buffers/cache:     730136    5192332
> Swap:      2097136    2097136          0
> 
> # cat /proc/sys/fs/file-max 
> 32768
> # cat /proc/sys/kernel/threads-max 
> 786432
> 
***snip***

> Our resident C expert thinks I am hitting some kind of max threads per
> user limit or max open files limit (I am running the milter under a
> non-privileged user).  So I wrote a little shell script that counts
> processes with 'ps -ef' and open files with 'lsof -i' and both never
> get above 400 and 300, respectively.  [Unfortunately my distro's
> version of ps does not support the -m flag for ps.]
> 
> Does anyone have any suggestions to resolv this, or has anyone
> experienced similar problems when using sync?
> 

As much as I can recall, the default stack size for a thread (in Linux
and elsewhere) is 2Mbytes. I am not sure what is libmilter setting here
- but if default - then 300 to 400 threads by themselves translate to
600 - 800 Mbytes of virtual memory. Then you also have your sendmail
processes running as well.

>From the kernel's standpoint, each thread is one process. If libmilter
is using the defaults, then each thread reserves 2M of virtual memory
space for its stack as well as your processes (sendmail & the rest).
This allocated space is just reserved; actual memory is allocated for
the stack on demand. But still, on a 32-bit processor, the total virtual
memory space available for the stacks is on the order of 1G, meaning
that more than 500 threads (and processes) will probably have a hard
time fitting in.

So despite of your ~6G of RAM - you may be hitting your (32-bit) system
limits here...

Just my 2c.

R.

On Thu, Oct 13, 2005 at 09:01:01PM +0000, Emmanuel Dreyfus wrote:

> > Actually - is there a global structure to which I can add a thread
> > count and then increment during any pthread_create (and decrement
> > during the destroy)?
> 
> You can add a global and change it in mlfi_connect (+1) and mlfi_close (-1)

Do I need to lock this variable, or is that handled automagically?


Thanks,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Fri, Oct 14, 2005 at 02:30:55PM -0400, Matthew S. Cramer wrote:
> > You can add a global and change it in mlfi_connect (+1) and mlfi_close (-1)
> 
> Do I need to lock this variable, or is that handled automagically?

It's not handled automagically, but I'm not sure we should care.

-- 
Emmanuel Dreyfus
manu@...

On Fri, Oct 14, 2005 at 07:56:45PM +0000, Emmanuel Dreyfus wrote:
> On Fri, Oct 14, 2005 at 02:30:55PM -0400, Matthew S. Cramer wrote:
> > > You can add a global and change it in mlfi_connect (+1) and mlfi_close (-1)
> > 
> > Do I need to lock this variable, or is that handled automagically?
> 
> It's not handled automagically, but I'm not sure we should care.

My threads from this never reach more than 30 or 40, which is
consistent with the amount of sendmail children I have running at any
given time.

I've added another global var and am now incrementing each time
sync_server starts and decrementing each time it returns.
Each time milter-greylist crashes it is preceeded by the
pthread_create errors of EAGAIN, always when trying to start a thread
for sync_server.

Thanks for the continued help.


Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Fri, Oct 14, 2005 at 12:54:49AM +0300, Ranko Zivojnovic wrote:

> From the kernel's standpoint, each thread is one process. If libmilter
> is using the defaults, then each thread reserves 2M of virtual memory
> space for its stack as well as your processes (sendmail & the rest).
> This allocated space is just reserved; actual memory is allocated for
> the stack on demand. But still, on a 32-bit processor, the total virtual
> memory space available for the stacks is on the order of 1G, meaning
> that more than 500 threads (and processes) will probably have a hard
> time fitting in.
> 
> So despite of your ~6G of RAM - you may be hitting your (32-bit) system
> limits here...

Thanks for the detailed explanation.  I understand this, but I am
having a hard time believing that all three of my servers are reaching
400 or 500 threads and processes.  We handle 30 messages at a time, so
rounding up the breakdown is:

    40 sendmail children
    40 milter-greylist threads for mail & syncing between 3 servers
    80 MIMEDefang processes
    80 other processes on the system

which is about what I see with ps: 240 processes.


Thanks,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Fri, Oct 14, 2005 at 04:15:20PM -0400, Matthew S. Cramer wrote:
> My threads from this never reach more than 30 or 40, which is
> consistent with the amount of sendmail children I have running at any
> given time.

Any Linux expert in the room who has an idea why thread creation would
fail? 

-- 
Emmanuel Dreyfus
manu@...

On Fri, Oct 14, 2005 at 04:15:20PM -0400, Matthew S. Cramer wrote:

> I've added another global var and am now incrementing each time
> sync_server starts and decrementing each time it returns.
> Each time milter-greylist crashes it is preceeded by the
> pthread_create errors of EAGAIN, always when trying to start a thread
> for sync_server.

I think I've made some progress.  I have 3 MX servers, and in 1 of
them I made the change above.  I was never able to get the milter to
run with sync enabled for more than a couple hours before crashing.  I
did this Friday before leaving for the weekend.  The server to which I
made the change did NOT crash all weekend, while the other two did.

Is it possible there is a race condition in sync_server, or some other
thread problem?  This is my current theory because to increment and
decrement my global var for counting sync threads, I set a mutex,
which may be resolving the real problem: the sync_server threads need
to wait on each other to finish.  There is a mutex in the code that
initiates a connection to another server but there wasn't one in the
function that handles the incoming connection.  Maybe you have to be
running more than 2 servers to see this problem?  Is anyone else using
more than 2?

I've moved my patched code over to the other 2 servers and will see
if they run all day without crashing, and will let the list know what
I find.  

My patches for reference:

bash-3.00$ diff milter-greylist.h milter-greylist-2.0.1/milter-greylist.h
96,99d95
< /* MSC */
< int syncers;
< /* MSC */
< 

bash-3.00$ diff milter-greylist.c milter-greylist-2.0.1/milter-greylist.c
586,592d585
< 
<       /* MSC */
<       extern int syncers;
< 
<       syncers = 0;
<       /* MSC */
< 

bash-3.00$ diff sync.c milter-greylist-2.0.1/sync.c
74,77d73
< /* MSC */
< pthread_mutex_t mutex1 = PTHREAD_MUTEX_INITIALIZER;
< /* MSC */
< 
826,834d821
<       /* MSC */
<       extern int syncers;
< 
<       pthread_mutex_lock( &mutex1 );
<       syncers++;
<       syslog(LOG_ERR, "Syncers: %d", syncers);
<       pthread_mutex_unlock( &mutex1 );
<       /* MSC */
< 
1003,1009d989
<       /* MSC */
<         pthread_mutex_lock( &mutex1 );
<         syncers--;
<         syslog(LOG_ERR, "Syncers: %d", syncers);
<         pthread_mutex_unlock( &mutex1 );
<       /* MSC */
< 



Optimistically,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

Hi,

>>>>> On Mon, 17 Oct 2005 11:07:17 -0400
>>>>> "Matthew S. Cramer" <mscramer@...> said:

mscramer> I think I've made some progress.  I have 3 MX servers, and in 1 of
mscramer> them I made the change above.  I was never able to get the milter to
mscramer> run with sync enabled for more than a couple hours before crashing.  I
mscramer> did this Friday before leaving for the weekend.  The server to which I
mscramer> made the change did NOT crash all weekend, while the other two did.

mscramer> Is it possible there is a race condition in sync_server, or some other
mscramer> thread problem?

How do you specify the peer?  If you specify it by a FQDN, please try
to change it to an IP address.  Your getaddrinfo(3) might not be
thread-safe.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

On Tue, Oct 18, 2005 at 12:18:31AM +0900, Hajimu UMEMOTO wrote:

> How do you specify the peer?  If you specify it by a FQDN, please try
> to change it to an IP address.  Your getaddrinfo(3) might not be
> thread-safe.

I specify IP addresses.


Thanks,

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Mon, Oct 17, 2005 at 11:07:17AM -0400, Matthew S. Cramer wrote:

> I think I've made some progress.  I have 3 MX servers, and in 1 of
> them I made the change above.  I was never able to get the milter to
> run with sync enabled for more than a couple hours before crashing.  I
> did this Friday before leaving for the weekend.  The server to which I
> made the change did NOT crash all weekend, while the other two did.

Dead end.  They are still crashing.

Not sure what the problem is, but it does not appear to be related to
too many threads.


Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Tue, Oct 18, 2005 at 04:52:49PM -0400, Matthew S. Cramer wrote:
> Dead end.  They are still crashing.
> 
> Not sure what the problem is, but it does not appear to be related to
> too many threads.

What about thread-safety problems?

-- 
Emmanuel Dreyfus
manu@...

Hi,

>>>>> On Wed, 19 Oct 2005 07:55:10 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Tue, Oct 18, 2005 at 04:52:49PM -0400, Matthew S. Cramer wrote:
> Dead end.  They are still crashing.
> 
> Not sure what the problem is, but it does not appear to be related to
> too many threads.

manu> What about thread-safety problems?

I noticed that pthread_detach() is not issued.  So, the resources of
the threads are not reclaimed.  The patch for this problem is attached
in this mail.  However, since I think that the mxsync threads are not
created so frequently, I'm not sure how it do for this issue.

Index: sync.c
diff -u -p sync.c.orig sync.c
--- sync.c.orig	Sat Oct 15 19:24:04 2005
+++ sync.c	Thu Oct 20 09:31:35 2005
@@ -560,19 +560,25 @@ sync_master_restart(void) {
 		    strerror(errno));
 		exit(EX_OSERR);
 	}
-	if (sync_master6.runs &&
-	    (error = pthread_create(&tid, NULL, sync_master,
-	    (void *)&sync_master6)) != 0) {
-		syslog(LOG_ERR, "Cannot run MX sync thread for IPv6: %s\n",
-		    strerror(error));
-		exit(EX_OSERR);
+	if (sync_master6.runs) {
+		if ((error = pthread_create(&tid, NULL, sync_master,
+		    (void *)&sync_master6)) != 0 ||
+		    (error = pthread_detach(tid)) != 0) {
+			syslog(LOG_ERR,
+			    "Cannot run MX sync thread for IPv6: %s\n",
+			    strerror(error));
+			exit(EX_OSERR);
+		}
 	}
-	if (sync_master4.runs &&
-	    (error = pthread_create(&tid, NULL, sync_master,
-	    (void *)&sync_master4)) != 0) {
-		syslog(LOG_ERR, "Cannot run MX sync thread for IPv4: %s\n",
-		    strerror(error));
-		exit(EX_OSERR);
+	if (sync_master4.runs) {
+		if ((error = pthread_create(&tid, NULL, sync_master,
+		    (void *)&sync_master4)) != 0 ||
+		    (error = pthread_detach(tid)) != 0) {
+			syslog(LOG_ERR,
+			    "Cannot run MX sync thread for IPv4: %s\n",
+			    strerror(error));
+			exit(EX_OSERR);
+		}
 	}
 }
 
@@ -696,6 +702,12 @@ sync_master(arg)
 			fclose(stream);
 			continue;
 		}
+		if ((error = pthread_detach(tid)) != 0) {
+			syslog(LOG_ERR, "incoming connexion from %s failed, "
+			    "pthread_detach failed: %s",
+			    peerstr, strerror(error));
+			exit(EX_OSERR);		/* XXX: recovery? */
+		}
 	}
 
 	/* NOTREACHED */
@@ -1093,6 +1105,10 @@ sync_sender_start(void) {
 	if ((error = pthread_create(&tid, NULL, 
 	    (void *(*)(void *))sync_sender, NULL)) != 0) {
 		syslog(LOG_ERR, "pthread_create failed: %s", strerror(error));
+		exit(EX_OSERR);
+	}
+	if ((error = pthread_detach(tid)) != 0) {
+		syslog(LOG_ERR, "pthread_detach failed: %s", strerror(error));
 		exit(EX_OSERR);
 	}
 	return;


Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

On Thu, Oct 20, 2005 at 03:57:11PM +0900, Hajimu UMEMOTO wrote:
> I noticed that pthread_detach() is not issued.  So, the resources of
> the threads are not reclaimed.  The patch for this problem is attached
> in this mail.  However, since I think that the mxsync threads are not
> created so frequently, I'm not sure how it do for this issue.

I've included that patch for the next release, but maybe it's worth waiting
to see if it fixes things or if there is another problem.

-- 
Emmanuel Dreyfus
manu@...

On Thu, Oct 20, 2005 at 03:57:11PM +0900, Hajimu UMEMOTO wrote:
> Hi,
> 
> >>>>> On Wed, 19 Oct 2005 07:55:10 +0000
> >>>>> Emmanuel Dreyfus <manu@...> said:
> 
> manu> On Tue, Oct 18, 2005 at 04:52:49PM -0400, Matthew S. Cramer wrote:
> > Dead end.  They are still crashing.
> > 
> > Not sure what the problem is, but it does not appear to be related to
> > too many threads.
> 
> manu> What about thread-safety problems?
> 
> I noticed that pthread_detach() is not issued.  So, the resources of
> the threads are not reclaimed.  The patch for this problem is attached
> in this mail.  However, since I think that the mxsync threads are not
> created so frequently, I'm not sure how it do for this issue.

I applied this patch and the other patch you posted regarding the
changes to pending.c for the memory usage issue.  milter-greylist has
been stable on all three of my MX servers for almost 24 hours, so this
definitely seems to have improved things!


Thanks!

Matt

-- 
Matthew S. Cramer <mscramer@...>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

On Fri, Oct 21, 2005 at 08:33:48AM -0400, Matthew S. Cramer wrote:
> I applied this patch and the other patch you posted regarding the
> changes to pending.c for the memory usage issue.  milter-greylist has
> been stable on all three of my MX servers for almost 24 hours, so this
> definitely seems to have improved things!

So let's roll out milter-greylist 2.0.2 with these changes

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz
MD5 (milter-greylist-2.0.2.tgz) = 3b407107760b09bd554fb7dc5c5a3854

Let me know if I missed a patch...

-- 
Emmanuel Dreyfus
manu@...

Hi,

>>>>> On Fri, 21 Oct 2005 12:47:23 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Fri, Oct 21, 2005 at 08:33:48AM -0400, Matthew S. Cramer wrote:
> I applied this patch and the other patch you posted regarding the
> changes to pending.c for the memory usage issue.  milter-greylist has
> been stable on all three of my MX servers for almost 24 hours, so this
> definitely seems to have improved things!

manu> So let's roll out milter-greylist 2.0.2 with these changes

manu> http://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz
manu> MD5 (milter-greylist-2.0.2.tgz) = 3b407107760b09bd554fb7dc5c5a3854

manu> Let me know if I missed a patch...

It seems okay to me.  Thanks.
It's minor issue but my previous patch that nukes p_addr from struct
pending was not merged.  If you decided not to merge it, it's okay to
me.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

On Sat, Oct 22, 2005 at 02:37:52AM +0900, Hajimu UMEMOTO wrote:
> It seems okay to me.  Thanks.
> It's minor issue but my previous patch that nukes p_addr from struct
> pending was not merged.  If you decided not to merge it, it's okay to
> me.

It's just that I missed it. Can you resend me the lost e-mail?
-- 
Emmanuel Dreyfus
manu@...

>>>>> On Sat, 22 Oct 2005 08:36:13 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> It's just that I missed it. Can you resend me the lost e-mail?

Okay, here it is.

Sincerely,

Hajimu UMEMOTO wrote:
> Hi,
> 
> 
>>>>>>On Wed, 19 Oct 2005 07:55:10 +0000
>>>>>>Emmanuel Dreyfus <manu@...> said:
> 
> 
> manu> On Tue, Oct 18, 2005 at 04:52:49PM -0400, Matthew S. Cramer wrote:
> 
>>Dead end.  They are still crashing.
>>
>>Not sure what the problem is, but it does not appear to be related to
>>too many threads.
> 
> 
> manu> What about thread-safety problems?
> 
> I noticed that pthread_detach() is not issued.  So, the resources of
> the threads are not reclaimed.  The patch for this problem is attached
> in this mail.  However, since I think that the mxsync threads are not
> created so frequently, I'm not sure how it do for this issue.

It seems that this patch has solved my "growing VmData" problem. Thanks!

Cheers,
Fredrik Nyberg

On Sun, Oct 23, 2005 at 12:33:06AM +0900, Hajimu UMEMOTO wrote:
> I found that a pending entry has a redundant p_addr field which holds
> string form of an IP address.  Since a pending entry has a p_sa field,
> I think it is elective.  Actually, an autowhite entry doesn't have
> such field.
> The following patch should reduce about 20 bytes per each pending
> entry.

But OTOH, it recomputes the string representation of the IP each time
we need it, instead of doingit only once. It's a speed vs memory trade. 
Are we sure it does not badly impact some setups where CPU is more scarce 
than memory? Anyone has an opinion on that topic? (I don't)

-- 
Emmanuel Dreyfus
manu@...

Hi,

>>>>> On Mon, 31 Oct 2005 10:09:27 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> On Sun, Oct 23, 2005 at 12:33:06AM +0900, Hajimu UMEMOTO wrote:
> I found that a pending entry has a redundant p_addr field which holds
> string form of an IP address.  Since a pending entry has a p_sa field,
> I think it is elective.  Actually, an autowhite entry doesn't have
> such field.
> The following patch should reduce about 20 bytes per each pending
> entry.

manu> But OTOH, it recomputes the string representation of the IP each time
manu> we need it, instead of doingit only once. It's a speed vs memory trade. 
manu> Are we sure it does not badly impact some setups where CPU is more scarce 
manu> than memory? Anyone has an opinion on that topic? (I don't)

As far as I saw, the only place to require a string form of an IP
address is the dumping, except error message outputs.
If it raises CPU usage considerably, it is better not to apply my
patch.
milter-greylist dumps struct pending and struct autowhite.  But,
struct autowhite doesn't have a field like p_addr.  This is a reason
why I made that patch.

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

On Mon, Oct 31, 2005 at 08:10:33PM +0900, Hajimu UMEMOTO wrote:
> As far as I saw, the only place to require a string form of an IP
> address is the dumping, except error message outputs.
> If it raises CPU usage considerably, it is better not to apply my
> patch.

I didn't saiy it would raise CPU usage, I said I wondered if it would.

> milter-greylist dumps struct pending and struct autowhite.  But,
> struct autowhite doesn't have a field like p_addr.  This is a reason
> why I made that patch.

That's a good point. 

-- 
Emmanuel Dreyfus
manu@...

> 
> As far as I saw, the only place to require a string form of an IP
> address is the dumping, except error message outputs.
> 

And there is a place in MX sync.
More precisely it is used in pending_del() for address comparison.
Of course, it may be replaced via comparing the content of the sockaddr structs.

Bests,
Attila

Hi,

>>>>> On Mon, 31 Oct 2005 14:33:03 +0100
>>>>> <attila.bruncsak@...> said:

attila> And there is a place in MX sync.

Ah, yes.  I forgot this, thanks.

attila> More precisely it is used in pending_del() for address comparison.
attila> Of course, it may be replaced via comparing the content of the sockaddr structs.

Yes, this comparison is replaced to use ip_equal() in my patch.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/