I'm testing NSD server performance on serveral zones and I found zonec
improvement point.
"zonec" needs large compile time for many(10000 or 1 million or more)
zones.
At each SOA RR, namedb_find_zone() is called and each SOA ownername
pointer is always compared with all zones->apex in this function.
As a result, this compare loop runs N(N-1)/2 times. (N = number of zones)
namedb_find_zone() processing time grows dominant as number of zones
grows.
To fix this, I added one member (zone_type pointer) to domain_type
and record zone pointer to APEX domainname.
This patch needs more memory space(number of ownernames * sizeof(pointer)).
But large number of zones case, it improves "zonec" performance.
This patch needs more memory space(number of ownernames * sizeof(pointer)).
But large number of zones case, it improves "zonec" performance.
If you are sure every zone is passed to zonec only once (the usual case) you can simply always start a new zone when a SOA record is encountered by applying the attached patch. This avoids the memory overhead of another pointer in domain_type.
In the future NSD will use a binary search instead here, so that should also solve this performance problem.
In the future NSD will use a binary search instead here, so that
should also solve this performance problem.
Don't forget to make sure it utilizes a balanced binary insertion
algorithm. As I discovered in my own adventures, it's really common
to end up with zone lists that are sorted in some kind of order when
they are automagically created out of a database or by some sort of
other script. Humans are deterministic creatures by nature, and the
world around them tends to get organized in some sort of way (even
unconsciously) and the resulting lack of randomness in the input data
stream tends to pessimize the performance of straight b-tree
algorithms.
From: Erik Rozendaal <erik@nlnetlabs.nl>
>
> This patch needs more memory space(number of ownernames * sizeof(pointer)).
> But large number of zones case, it improves "zonec" performance.
If you are sure every zone is passed to zonec only once (the usual case)
you can simply always start a new zone when a SOA record is encountered
by applying the attached patch. This avoids the memory overhead of
another pointer in domain_type.
I know this. I found this problem a half year ago(changes between
nsd-2.1.1 and 2.1.2), someone proposed same as this patch in my
company.
But I cannot sure always usual and this should be detected. Then, I
added one bitfield to domain_type and check it and this patch does not
cost memory usage.
Is this patch acceptable?
# operators dislike unofficial patches.
In the future NSD will use a binary search instead here, so that should
also solve this performance problem.
# I believe this namedb_find_zone() search is useless.
But I cannot sure always usual and this should be detected. Then, I
added one bitfield to domain_type and check it and this patch does not
cost memory usage.
Is this patch acceptable?
Yes, I've put it into CVS for the next release. Thanks!