Hosting many domains as a secondary: parallelizing "nsdc update"

Our ns2.nic.fr is a secondary name server for 458 domains.

Many of them are broken and axfr times out.

Running "nsdc update" takes a *long* time, partly because it runs
sequentially. This is a real issue with nsd.

Also, some domains are more important than others (".fr" :slight_smile: and we
would like to process them first.

Therefore, I suggest the following solution:

1) from nsd.zones, produce a Makefile
2) run this Makefile, either with "make fr" to update only one domain,
or with "make -j N" if you want to run N axfr in parallel (yes, it
requires GNU make).

Advices? I attached the script get-one-zone, used by the Makefile to
retrieve one zone, and the script which converts nsd.zones to a
Makefile.

It seems it works.

(attachments)

zones2make.py (1.55 KB)
get-one-zone (2.5 KB)

a message of 214 lines which said:

Therefore, I suggest the following solution:

Implemented today, we'll see if our Russian or Dutch colleagues scream
:slight_smile:

Advices?

Here is the suggested patch against nsd 2.3.5. We would like to see it
integrated.

(attachments)

nsd-parallel-make.patch (11.1 KB)

Stephane Bortzmeyer wrote:

a message of 214 lines which said:

Therefore, I suggest the following solution:

Implemented today, we'll see if our Russian or Dutch colleagues scream
:slight_smile:

Advices?

Here is the suggested patch against nsd 2.3.5. We would like to see it
integrated.

Hi Stephane,

Thanks for the bugreports and the patch. Excellent stuff; I have
incorporated bugfixes for the locking bugs you found and the patch is
added to the nsd contrib folder (I have adapted to patch to work with
the current svn, and added a small readme/howto for it). This will be in
the next point release.

I did not incorporate the patch into the mainline NSD 2.x code. Partly
because it is a complicating feature, but mostly because with the
upcoming NSD 3 you can get the same functionality with its AXFR/IXFR
feature.

NSD3 can perform AXFR / IXFR transfer requests in parallel out of the
box. Currently it is '-j 10' (tweaked with the XFRD_MAX_TCP constant).
NSD3 does not use gnu make for that, it starts a separate process for
the transfers, that sleeps in a select() statement for most of the time.
At a timeout or a notify message it will wake up and perform a zone
transfer for the zone (with IXFR and TSIG support). So it has only one
process that does all the transfers.

The new feature keeps track of the time as well, so that zones are
refreshed according the times from the SOA record. You can send notify
messages to the server to force zone updates.

NSD 3 is in testing now, you can get it from the subversion repository
if you are interested.

Best regards,
   Wouter