Pawel Krzesniak, from NASK (".pl" registry), just announced on a CENTR
mailing list that ".pl" now uses IXFR to speed up the transfer of new
data to the secondaries. And he is very happy about the results.
At the present time, nsd, acting as a server, refuses IXFR. Acting as
a client, nsd-xfer has no IXFR support and, since it is completely
separated from the name server, it seems difficult to add it.
Are there any plans to add IXFR in nsd? Is it realistic?
[On 15 Feb, @ 22:14, Stephane wrote in "IXFR and nsd ..."]
At the present time, nsd, acting as a server, refuses IXFR. Acting as
a client, nsd-xfer has no IXFR support and, since it is completely
separated from the name server, it seems difficult to add it.Are there any plans to add IXFR in nsd? Is it realistic?
some time ago there was an offlist discussion about this. IIRC the
conclusions where:
o if a secondary has no access to the actual TXT zone file, but only
the database your stuck.
o it was deemed that this was tricky and heading to creeping
feature-ism.
IMO if the actual zone transfers get too large, why not use something
better suited like rsync?
grtz Miek
rsync assumes you have a single zonefile in text format available, if
nsd has to parse it.
What is recently happening is that the TLDs are starting to not spool
the entire zone, but only send dynamic updates to the master from their
authoritative source.
I think it is becoming increasingly critical for larger TLDs to have
IXFR, now that they are getting used to it, and subsequently their
users.
[Quoting Robert Martin-Legene, on Feb 16, 10:09, in "Re: IXFR and nsd ..."]
rsync assumes you have a single zonefile in text format available, if
nsd has to parse it.
When we set the the requirements for NSD, we did assume that the
zone-information is available in standard zone-file format from
which the NSD-database is built, which in turn is used by our
(mean and lean) daemon process.
It has been discussed a few times, but sofar we feel that this
was and still is a good decision.
What is recently happening is that the TLDs are starting to not spool
the entire zone, but only send dynamic updates to the master from their
authoritative source.
Supporting dynamic updates was a specific non-requirement for NSD at
the time. Later it turned out, that having the daemon to switch
NSD-database was even faster (and certainly simpler and thus less
prone to errors) than trying to update the NSD-database in flight.
Since building the NSD-database is done seperately from running the
daemon we think that dealing with dynamic updates outside the
daemon was and still is a very good decision.
I think it is becoming increasingly critical for larger TLDs to have
IXFR, now that they are getting used to it, and subsequently their
users.
Although we are still not sure that AXFR/IXFR is the best way to
distribute and maintain zone-information between the various
nameservers for important/large zones (like root and TLD), we are
listening to these requests.
If we do implement IXFR support, it will be done, like our AXFR
support, outside the daemon process. However, we have no plans
(and thus no timescale) on implementing IXFR support yet.
Regards,
-- ted
There is talk in the strategy group for .SE to do the same, and here, while
my NSD machines are at the forefront in terms of availability, DNSSEC
features and speed, they lack in functionality for IXFR. Given the 6 months
horizon on DNSSEC deployment ;-), we're looking at larger zones.
And, because Some Registries bend over backwards to accomodate
domain-name-holder fsckups and push dynupdates every 5 minutes (I wonder
when customers will press marketroids into defeating caching...), the
marketing frontends see some increasing pressure to do this. For SE, an
AXFR per hour or so is not a problem given our network infrastructure, but
it would be pushing it to go much further.
See this as a weak prayer for IXFR. I will try to get some attention to the
problem, but don't hold your breath...
O.k. 
Well I do, so this is a quick check to make sure I'm not
duplicating effort. We're hosting a ~= 6million record large zone which
we can't get rsync access to, and because it is US-hosted, the AXFR's
are now taking 4 hours, so it's worth my while to code this It's an
RBL zone, and the information is relatively time-sensitive.
My current project plan is to implement something that does:
ixfr-client zonefile [origin] [nameserver]
and outputs a zonefile. From my research on the IXFR standard
this is certainly implementable, and the rough idea is:
1. Parse the zonefile into memory
2. Formulate an IXFR query, extracing the SOA, origin
and target namerserver(s) from the zonefile - if
not supplied from the command line.
3. Parse the response and consolidate two lists:
records to be added
records to be removed
OR if the SOA we have was too old, just a list of
the new records (ala AXFR).
4. Apply these changes to the memory-representation of
the original zonefile
5. Output a zonefile
I can't see any major technical barriers anyway, I just have to code it
all. From my preliminary research it looks like there are no good
generic zonefile parsing libraries, the closest I've found is perl's
DNS::Zoneparse, and that doesn't come very close, so it looks like I'll
be writing it in C and will use the zparser and zonec code from NSD,
since I'm more familiar with NSD's code-base than anything else.
Anyway, would the above prove of general use to people here? would it
satisfy your requirements? Is anyone interested in helping?
[Quoting Colm MacCarthaigh, on Mar 3, 12:12, in "Re: IXFR and nsd ..."]
O.k.
duplicating effort. ...
I looked at your rough idea, it's identical to what
we have in mind, but haven't started yet.
Thanks and regards,
-- ted
I know it's not the easiest thing to do, since it's an external
application, but with big zones with frequent updates (your RBL list
being a candidate) won't you be spending most of your CPU time
continuously compiling the new zone with zonec?
Would it be possible to copy the DB-file and do the IXFR-changes in that
file directly, and then tell NSD to use the new file once done?
I know it's not the easiest thing to do, since it's an external
application, but with big zones with frequent updates (your RBL list
being a candidate) won't you be spending most of your CPU time
continuously compiling the new zone with zonec?
Yep, it takes us about 11 minutes to compile our zone, and that's on
some pretty beefy hardware with more than enough memory. But the one
updates every 3 hours, and it's currently taking more than that to
transfer - so it is at least an improvement.
Would it be possible to copy the DB-file and do the IXFR-changes in that
file directly, and then tell NSD to use the new file once done?
It won't really be that much quicker, it takes almost as much time to
read in the db file as it does to parse a zone-file, NSD it set-up for
run-time efficiency - not speedy compiles (but then nothing is, for the
type of sizes we're dealing with). I wrote an incremental compiler for
NSD a long time ago, which cached db files for each zone and only
recompiled the ones which had a newer timestamp than their db files and
then read all of the db files and output a master db file - but it
actually ended up taking longer to run than zonec itself, the vast
majority of time is just calls to read(), so it got abandoned.