Milter-greylist

Hi 

I am working on nsupdate support in milter-greylist, so that an action
clause in an ACL can trigger a DNS update, which enables milter-greylist
to feed a DNSRBL based on ACL conditions.

That first part works fine, but I thought about handling expiration of
added DNS record. It seems there is no way to tell the DNS that an added
entry should expire. That would suggest milter-greylist sould manage it,
which leads to a lot of complexity. 

What people think about it? Is expiration actually useful, and how to
implement it?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On 2013-05-19 07:13, manu@... wrote:
> Hi
>
> I am working on nsupdate support in milter-greylist, so that an action
> clause in an ACL can trigger a DNS update, which enables milter-greylist
> to feed a DNSRBL based on ACL conditions.
>
> That first part works fine, but I thought about handling expiration of
> added DNS record. It seems there is no way to tell the DNS that an added
> entry should expire. That would suggest milter-greylist sould manage it,
> which leads to a lot of complexity.

Well, I used dynamic DNS in our LAN to point a service name to
recently-working instances of the service (i.e. "proxy" points
to several SQUID instances if they work), and the script which
does this just tests the service and calls the nsupdate binary:

# case ...  0) # OK
                 ( echo "server $DNSMASTER"
                   echo "zone $DNSZONE"
                   echo "update add $DNSZONE. $DNSTTL A $IP"
                   echo "" ) | nsupdate
                 ;;
             *) # FAIL
                 ( echo "server $DNSMASTER"
                   echo "zone $DNSZONE"
                   echo "update delete $DNSZONE. A $IP"
                   echo "" ) | nsupdate

This is on Solaris, but AFAIK the nsupdate part is a standard
BIND one (I did have to update BIND on some older DNS servers
to have it working as I needed). As you can see above, there
is a possibility to set the TTL for an added entry, which I
believe should override the one set in the zone's SOA record.

Your other solution choice is (if you own the DNS zone) to
set those SOA fields to shorter timeouts...

HTH,
//Jim Klimov

On Sun, May 19, 2013 at 11:17 AM, Jim Klimov <jimklimov@...> wrote:
> On 2013-05-19 07:13, manu@... wrote:
> This is on Solaris, but AFAIK the nsupdate part is a standard
> BIND one (I did have to update BIND on some older DNS servers
> to have it working as I needed). As you can see above, there
> is a possibility to set the TTL for an added entry, which I
> believe should override the one set in the zone's SOA record.
>
> Your other solution choice is (if you own the DNS zone) to
> set those SOA fields to shorter timeouts...

But TTL is for the client to decide if it should refresh it's cache or
not (use cache or ask server again), it doesn't mean the record will
expire on the server which I believe is what manu means.

/peter

Peter Bonivart <shuttlebox@...> wrote:

> But TTL is for the client to decide if it should refresh it's cache or
> not (use cache or ask server again), it doesn't mean the record will
> expire on the server which I believe is what manu means.

Yes, you are right. TTL is no expiration mechanism. 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sun, May 19, 2013 at 1:03 PM,  <manu@...> wrote:
> Yes, you are right. TTL is no expiration mechanism.

Maybe the addition could be logged and the timestamp could be used by
an external script to remove old entries.

/peter

Peter Bonivart <shuttlebox@...> wrote:

> > Yes, you are right. TTL is no expiration mechanism. 
> Maybe the addition could be logged and the timestamp could be used by
> an external script to remove old entries.

That may be the easiest way. Using the greylisting database requires
heavy refactoring, as the stored object is quite different from the
current tuple.

One caveat for the log file: how do we resume operations? We lookup all
addresses in the log until we find one not registered? And we need to
clear older entries after some time, but how can it be done efficiently?
(that is without moving the data within the file toward start)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sun, May 19, 2013 at 4:24 PM,  <manu@...> wrote:
> One caveat for the log file: how do we resume operations? We lookup all
> addresses in the log until we find one not registered? And we need to
> clear older entries after some time, but how can it be done efficiently?
> (that is without moving the data within the file toward start)

Don't log the addition until you have actually done the nsupdate op,
after that it's BINDs responsibility and it uses journaling for the
transactions so it should be safe to restart.

If the log file is produced by syslog a contributed script can parse
it using a marker, extracting timestamp and addresses. Number of hours
for removal should be configurable. Even simpler would be to combine
it with logrotate and have a script remove all entries in the not
current log file. Up to the user to set logrotate intervals.

/peter

Peter Bonivart <shuttlebox@...> wrote:

> If the log file is produced by syslog a contributed script can parse
> it using a marker, extracting timestamp and addresses. Number of hours
> for removal should be configurable. Even simpler would be to combine
> it with logrotate and have a script remove all entries in the not
> current log file. Up to the user to set logrotate intervals.

Another idea: for each A record stored, I can add a TXT record for the
current date. For instance, for today may 20th, I blacklist 192.0.2.1
and 192.0.2.2. I would have in my zone file:

1.2.0.192.bl.example.net IN A 127.0.0.1
2.2.0.192.bl.example.net IN A 127.0.0.1
20130520.bl.example.net IN TXT 1.2.0.192.bl.example.net 
20130520.bl.example.net IN TXT 2.2.0.192.bl.example.net 

That way I just need to perform a DNS lookup on current date to get all
the records I need to cleanup.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On 2013-05-19 11:37, Peter Bonivart wrote:
> But TTL is for the client to decide if it should refresh it's cache or
> not (use cache or ask server again), it doesn't mean the record will
> expire on the server which I believe is what manu means.

Hmm... I guess you're right. Still, it can be part of the expiration
mechanism - if the server has the blacklist entry removed by the time
of TTL re-request, the client would refresh its cache and see that
the target IP is no longer blacklisted.

I don't think there's a DNS notion of entry expiration. There's zone
expiration, which means that after some time that it has been cached
and couldn't be refreshed from source, it should no longer be trusted,
and queries via this cached data should begin to fail. This allows to
"put to rest" and discard unmaintained DNS zones which would otherwise
pollute the internet forever.

//Jim