Hello All,
Should we expect an increase in the memory footprint for NSD4, and if so, how much? Also, what about the storage size of the db file?
Thanks,
W
Hi Will,
Hello All,
Should we expect an increase in the memory footprint for NSD4, and
if so, how much? Also, what about the storage size of the db
file?
Yes, NSD4 uses more memory to make it handle zone changes faster.
If you compile --without-nsec3, the memory size is the same as NSD3.
But --with-nsec3 (the default), the memory size is increased. There
is extra information to enable IXFR for NSEC3 zones to be handled very
quickly (rehashing and precompilation). For a 64bit machine this is
about 104 bytes more per domain name for a non-nsec3-zone, and about
360 bytes more per domain name for a zone that uses nsec3 (this is an
estimate).
For 1 million domain names, that works out to 99 Mb memory used more
than NSD3, or 343 Mb more when the zone is NSEC3-signed. (this
compares to NSD3 with full prehash enabled (the default)). For 32bit
compilation(gcc -m32), 78Mb or 200Mb, it is mostly pointers.
The storage size of the db file is significantly larger and adds to
the memory usage, because it is random-access. This enables IXFRs to
be serviced faster and removes nsd-patch. The file is memory mapped
(mmap(2) system call), and thus adds to the virtual memory size. It
is not accessed during network service, but the reload process of NSD
accesses it when processing zone transfers. The database size is
several Gb for a TLD-sized zone. The database size is not constantly
growing (apart from your zone growth, of course), it is compacted when
space becomes freed.
Although the nsd.db counts with it mmap into the memory you see in the
process table, it is not necessary to have sufficient memory for that
(unless you want zone transfers to be handled quickly). With not
enough memory it'll simply read and write to disk (this is normal mmap
behaviour) when processing a zone transfer. The processes that
service port 53 do not access the mmap-area, so do not block on disk
access.
You can simply subtract the nsd.db size from the VSZ to get the size
you need for network service. nsd4 also reports memory and disk usage
with nsd-control stats.
Best regards,
Wouter
Hi,
We are currently looking at memory footprint of NSD4 again, and we
think substantial improvements are possible at a small cost. This is
implemented with changes in the data structures.
Best regards,
Wouter