Milter-greylist

Wanted to try out version 4 on my "test production" machine and decided to 
try out GeoIP as well. Downloaded latest source for libGeoIP and started 
compiling. When compiling the regionname object cc1 starts to grab memory 
infitely. It least it seems that it is not going to stop, I stopped it 
around 750 MB as it seemed largely inappropriate to me that it should need 
that much memory for compilation (and I was about running out of memory). 
This is a SuSE 9.0 system. Has anyone seen this problem?

Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

Kai Schaetzl wrote:
 > Wanted to try out version 4 on my "test production" machine and decided to 
 > try out GeoIP as well. Downloaded latest source for libGeoIP and started 
 > compiling. When compiling the regionname object cc1 starts to grab memory 
 > infitely. It least it seems that it is not going to stop, I stopped it 
 > around 750 MB as it seemed largely inappropriate to me that it should need 
 > that much memory for compilation (and I was about running out of memory). 
 > This is a SuSE 9.0 system. Has anyone seen this problem?

What version of gcc is that?  (Type "gcc -v".)

Version 4.2 has an efficiency problem (some call it bug)
that it consumes large amounts of memory for certain kinds
of source files.  In particular, it requires about 1 GB
of memory (RAM + swap) for one file in the Xorg tree when
the optimization level is >= 1.  That problem (and several
others) is fixed in gcc 4.2.1.

Bottom line:  Try updating your gcc, and/or try compiling
that particular file with a lower optimization level
(-O0 if everything else fails).

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"C++ is to C as Lung Cancer is to Lung."
        -- Thomas Funke

Oliver Fromme wrote on Thu, 2 Aug 2007 10:15:55 +0200 (CEST):

> Bottom line: Try updating your gcc, and/or try compiling
> that particular file with a lower optimization level
> (-O0 if everything else fails).

Thanks a lot for the recommendation. Setting CFLAGS=-O1 or -O0 gets it 
going. I looked in the Makefile and configure.status, but didn't find a 
way to apply that flag only for compiling this object file. Anyway, is it 
correct that this is only a compile-time optimization and not a runtime 
optimization of the produced binary? So, the result is just the same, but 
may take a bit longer to compile?

I checked on another SuSE 9.0 system and on older systems and was able to 
verify that gcc works fine on the older systems, but not on 9.0. This is 
version gcc-3.3.1-29 and it looks very much like it is affected by the 
same bug you mentioned for 4.2.

Compiling milter-greylist 4.0a6 with libGeoIP works after this just fine.

Now, how do I make use of it in milter-greylist? Is the following correct?

geoipdb=/path/to/it
racl greylist geoip CN KR delay 1d

Is this correct for greylisting clients from China and Korea for one day?

Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

Kai Schaetzl wrote on Thu, 02 Aug 2007 16:14:25 +0200:

> geoipdb=/path/to/it
> racl greylist geoip CN KR delay 1d

Using something like
list "badcountries" geoip { CN }
racl greylist list "badcountries" delay 1d geoipdb "/path/to/GeoIP.dat"

still doesn't work. I get a syntax error already with the list definition 
and when starting the milter without these lines I get "GeoIP is not 
available" in the log. Does this suggest it can't utilize GeoIP although 
compiled in? I compiled it with libGeoIP and the make output showed that 
it gets compiled in and succeeded.

Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

The make doesn't run an ldconfig, so I had to run that manually. Now the 
provided tool geoiplookup works. But milter-greylist still reports "GeoIP 
is not available". Do I have to recompile it after the ldconfig although 
the earlier compilation didn't show an error?

(Yahoo Groups seems to be slow today, none of my earlier mails made it 
thru yet.)

Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

Kai Schaetzl wrote:
 > Oliver Fromme wrote:
 > 
 > > Bottom line: Try updating your gcc, and/or try compiling
 > > that particular file with a lower optimization level
 > > (-O0 if everything else fails).
 > 
 > Thanks a lot for the recommendation. Setting CFLAGS=-O1 or -O0 gets it 
 > going. I looked in the Makefile and configure.status, but didn't find a 
 > way to apply that flag only for compiling this object file. Anyway, is it 
 > correct that this is only a compile-time optimization and not a runtime 
 > optimization of the produced binary? So, the result is just the same, but 
 > may take a bit longer to compile?

No, the optimization is for the code generation backend.
So the resulting binary _might_ be slower.  The difference
between -O2 and -O1 usually isn't large (I've even seen
programs that were actually faster with -O1), but -O0 is
often noticeably slower.

Compiling just that one file with different optimization
should be easy.  Basically it works like this:

Build the whole app with the standard optimization and
watch for the file causing the trouble (where it hangs).
Abort with ^C, change to the proper directory if necessary,
copy the compiler call to your shell prompt, replace the
optimization option with the lower one and execute it
manually.  Then continue building.  The "make" tool will
recognize that the object file is already up to date, and
it will continue compiling the rest of the files with the
standard optimization.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"It combines all the worst aspects of C and Lisp:  a billion different
sublanguages in one monolithic executable.  It combines the power of C
with the readability of PostScript."
        -- Jamie Zawinski, when asked: "What's wrong with perl?"

Kai Schaetzl <maillists@...> wrote:

> list "badcountries" geoip { CN }
> racl greylist list "badcountries" delay 1d geoipdb "/path/to/GeoIP.dat"

Almost. The geoipdb location is a global setting

Something like this should do it (not tested)

geoipdb "/path/to/GeoIP.dat"
list "badcountries" geoip { "CN" "KR" }
racl greylist list "badcountries" delay 1d msg "%C wait one day!"


-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

wrote on Fri, 3 Aug 2007 14:29:47 +0200:

> geoipdb "/path/to/GeoIP.dat"

Ok, that got me going, thanks. I looked in the source and that "GeoIP is 
not available" is issued when no database is specified.

Now let's see how long it takes yahoogroups to accept this message ...



Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

Oliver Fromme wrote on Fri, 3 Aug 2007 09:00:35 +0200 (CEST):

> No, the optimization is for the code generation backend.
> So the resulting binary _might_ be slower.  The difference
> between -O2 and -O1 usually isn't large (I've even seen
> programs that were actually faster with -O1), but -O0 is
> often noticeably slower.

Ok, I guess I will then just let it stay this way as I used O1. I can try 
that "compilatus interruptus" with the next 4.0 alpha build. Thanks!



Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com