Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Message

Makefile fix for parallelized build of milter-greylist

2013-01-20 by Jim Klimov

Hello all,

   I have been fixing milter-greylist to build on my farm of assorted
Solaris machines (from Sol8 to OI), and a generic problem popped up:
if I execute a parallel build (i.e. "gmake -j all") I am almost always
receiving an error (except on one very fast computer). Tracing the
problem got me to the POSIX-based usage of yacc/bison, which causes
an output file to have a fixed name. When two build jobs write into
it in parallel - bad things happen. Maybe this is also augmented by
my boxes using the same build homedir over NFS (taking turns, of
course - but lags/unsyncs can be assumed to happen in comparison
to local HDD filesystems).

   So, I came up with a solution which works for me with GNU gmake and
Sun CCS make, but before trying to commit it into the source repo it
should be checked that this solution does not break builds (sequential
and parallel) on other OSes and build platforms (i.e. with other make
programs). Solution is attached, a patch that delivers fixes to the
Makefile.in template and a minor one to configure.ac (run "autoconf"
to build the new configure script).

   Ultimately I've made the attached patches to reinvoke the make
in non-parallel mode for those targets while not having to rebuild
milter-greylist itself which no longer directly depends on those
object files - by the virtue of touching a flag-file which for
make is an okay dependency.

   The sequential invokation of make is a Makefile variable MAKESEQ
which should ideally be set by configure - contributions of detection
for different make programs are welcome. So far value "make -j 1"
is hard-coded, and an example of another (which also works for gmake)
which zeroes out MAKEFLAGS is offered in comments. The latter is
not gracious IMHO, because other flags could be in use - like VPATH,
etc. With such a simple hack they would be lost.

   Also, while my tests ran over NFS-mounted home, there were some
lags which caused the first run to fail (sync/timing issues) while
the second sequential one passed ok. I don't think this can be worked
around well, had this for years - sequential builds over NFS also
fall prey to this issue, though much more rarely.

   One way or another, I also made a "rebuild" target which does
"clean" and then reinvokes make to build all - in original mode
(may be paral or seq as user requests) and if that fails, retries
in seq mode. In part this target is to test the routine, and may
be practical - i.e. if sources to those objects which are now
hidden by the lockfile are updated, a "rebuild" may ease the...
um... rebuilding of the project properly

   Also I added "sync || true" to fence off some parts which broke
for me over NFS (i.e. cleans must complete before builds begin).

   I've tested this with gmake and Sun (CCS) make, works ok for me
for local and NFS builds.

   If possible, this should be tested with more "make" programs and
perhaps the "sync" call should also be detected by configure - and
invokations skipped if the program is not present. Likewise, the
exact spelling of make-sequential invokations (-j1, -j 1, something
else) should also be configure-able, which they are not now in my
patch.

HTH, thanks in advance for verifications, and good luck,
//Jim Klimov

Attachments

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.