Folk,
We recently got a query from somebody in the industry asking how Unbound compares to Windows Resolver performance in terms of qps and CPU (memory is under user control).
All our benchmarking is Unix based and we never really did a comparison ourselves. Is there somebody on this list that has performed such measurements, or knows of a reference?
Thanks!
—Olaf Kolkman
NLnet Labs.
Could I ask if a comparison of Unbound vs BIND has been documented for
a Unix platform?
Thanks,
Maurice
One example, something done by a 3rd party: http://www.iepg.org/2010-11-ietf79/iepg79-mglt.pdf
In general recursive name servers are hard to compare because of how caching affects performance.
—Olaf
Hello Olaf,
Olaf Kolkman wrote:
Folk,
We recently got a query from somebody in the industry asking how
Unbound compares to Windows Resolver performance in terms of qps and
CPU (memory is under user control).All our benchmarking is Unix based and we never really did a
comparison ourselves. Is there somebody on this list that has
performed such measurements, or knows of a reference?
I have a working setup where I could test Unbound vs. MS DNS (the setup
where I do regression tests on the Unbound packages I compile) and would
be very interested in the results as well. The machines are not too
powerful (dual-core with 8GB RAM) compared to todays standards.
I'm pretty busy the next two weeks, but could do the tests later in June
(if that helps).
Is the query about comparing Unbound on Windows with MS DNS, or
comparing Unbound on Unix/Linux with Win DNS?
Carsten
I'd be interested in the results comparing the two on a similar/identical platform. In other words, should I uninstall Microsoft DNS and install unbound on the same system (where unbound is otherwise a reasonable fit)
Obviously it's not a suitable replacement for Active Directory driven DNS. However, even here, there's an interesting performance question: Is it worth installing unbound and forwarding Microsoft DNS to unbound, or is it better to let Microsoft DNS perform it's own resolution?
Hello Olaf,
Olaf Kolkman wrote:
Folk,
We recently got a query from somebody in the industry asking how
Unbound compares to Windows Resolver performance in terms of qps and
CPU (memory is under user control).All our benchmarking is Unix based and we never really did a
comparison ourselves. Is there somebody on this list that has
performed such measurements, or knows of a reference?I have a working setup where I could test Unbound vs. MS DNS (the setup
where I do regression tests on the Unbound packages I compile) and would
be very interested in the results as well. The machines are not too
powerful (dual-core with 8GB RAM) compared to todays standards.
Perfect!!!
I'm pretty busy the next two weeks, but could do the tests later in June
(if that helps).
Hey, I am the one asking a favor 
Is the query about comparing Unbound on Windows with MS DNS, or
comparing Unbound on Unix/Linux with Win DNS?
The former.
—Olaf
Hello Dave,
Dave Warren writes:
Is the query about comparing Unbound on Windows with MS DNS, or
comparing Unbound on Unix/Linux with Win DNS?I'd be interested in the results comparing the two on a
similar/identical platform. In other words, should I uninstall Microsoft
DNS and install unbound on the same system (where unbound is otherwise a
reasonable fit)
That is what I will test next week, I will post the results here (and
the test setup).
Obviously it's not a suitable replacement for Active Directory driven
DNS.
why not? It is best practice to separate DNS resolver (caching DNS
server like Unbound) and authoritative Server. While WinDNS can be used
in both functions, it makes a good resilient and manageable DNS design
to separate the DNS server functions on dedicated machines.
Unbound will nicely work as an secure DNSSEC validating resolver,
resolving Internet names and also (possible) local Active Directory
names that are stored on WinDNS AD integrated servers.
However, even here, there's an interesting performance question: Is
it worth installing unbound and forwarding Microsoft DNS to unbound, or
is it better to let Microsoft DNS perform it's own resolution?
Forwarding is (today) probably almost always slower than direct name
resolution (and more complicated and brittle), unless you are connected
to the Internet with a slow link. I recommend to not use forwarding
unless there are very special conditions.
Unbound as a direct resolver might be faster than having WinDNS as a
direct resolver.
Dave Warren writes:
Obviously it's not a suitable replacement for Active Directory driven
DNS.why not? It is best practice to separate DNS resolver (caching DNS
server like Unbound) and authoritative Server. While WinDNS can be used
in both functions, it makes a good resilient and manageable DNS design
to separate the DNS server functions on dedicated machines.
In general, I agree that it makes sense to split authoritative and resolver roles. However, in the case of Windows and Active Directory, Active Directory is built under the assumption that your DNS servers accept AD authenticated dynamic updates, both from AD itself and from clients, so it's best practice to only specify Microsoft DNS servers for Active Directory domain controllers, member servers and workstations when possible.
While you can do it via other methods (setting up AD's entries manually or forwarding the appropriate zones), it takes a lot of head-banging to get everything working and if you mess it up, the effects are subtle and intermittent since parts of Windows will fall back on broadcasts and other unreliable methods, and therefore will sometimes work even with DNS misconfigured.
Also keep in mind that Microsoft's authoritative DNS is multi-master and site-aware (so a machine registered in the current site will be immediately available in DNS to the current site, but might take time to propagate to other physical sites in the same DNS zone, balancing the need for quick updates vs keeping the number of updates between sites reasonable)
My theory is that each site (physical location as well as Active Directory site/subnet) would have one unbound server that performs internet resolution, with multiple AD servers that forward to the unbound server.
Unbound will nicely work as an secure DNSSEC validating resolver,
resolving Internet names and also (possible) local Active Directory
names that are stored on WinDNS AD integrated servers.
Microsoft DNS's DNSSEC support is limited at best, and it has no pre-fetch support at all, so I'd like to use unbound for primary DNS resolution. However, hosting Active Directory on anything but Microsoft's DNS is outside of best practices for Active Directory.
However, even here, there's an interesting performance question: Is
it worth installing unbound and forwarding Microsoft DNS to unbound, or is it better to let Microsoft DNS perform it's own resolution?Forwarding is (today) probably almost always slower than direct name
resolution (and more complicated and brittle), unless you are connected
to the Internet with a slow link. I recommend to not use forwarding
unless there are very special conditions.Unbound as a direct resolver might be faster than having WinDNS as a
direct resolver.
It might. If so, I'd like to know how much faster or slower the servers are on their own, but also how much overhead is involved if Microsoft's DNS sits in the middle to see if complying with best practices is appropriate, or if there's a technical justification to go with a more complicated setup.
I have the impression that Microsoft DNS isn't particularly speedy, but I have not actually attempted to benchmark it since Windows 2003 vs an appropriate era BIND. At the time, BIND was faster, but only slightly, but since this design effectively allowed for a shared cache, the real-world performance was significantly improved.
My guess is that having several Microsoft DNS servers forward to a single unbound server which does resolution of all non-local zones will ultimately be slightly faster than having multiple Microsoft DNS servers do the work themselves, but even if it's ultimately slightly slower, gaining the benefits of Unbound's DNSSEC validation probably make it worthwhile. But if it's a lot slower, I would definitely be open to other configurations.
In general, I agree that it makes sense to split authoritative and resolver
roles. However, in the case of Windows and Active Directory, Active
Directory is built under the assumption that your DNS servers accept AD
authenticated dynamic updates, both from AD itself and from clients, so it's
best practice to only specify Microsoft DNS servers for Active Directory
domain controllers, member servers and workstations when possible.
First a caveat - all of my clients and experience (after my big iron
days that is) are small businesses. Most are non-AD but a couple of
them do have AD domains running on Microsoft servers (no giant
Forests, one domain). One is an inherited account and the other needed
to run software that required an AD. Not my preference (which is
Linux/BSD servers), but they work.
So file this under some guys opinion and probably not applicable to
your environment.
First, unless a box is a server there's no pressing need for it to
have DNS entry. Nice, can be helpful, but not absolutely needed in
most cases.
My theory is that each site (physical location as well as Active Directory
site/subnet) would have one unbound server that performs internet
resolution, with multiple AD servers that forward to the unbound server.
At one site I run the Unbound resolver/cache on an OpenBSD box
configured with stub-zones for the AD domain (forward and reverse).
All clients are configured to query this Unbound box. The AD server
forwards to Unbound anything it is not authoritative for.
The important thing this accomplishes (at least in my paranoid mind)
is that it removes the AD from direct Internet access for DNS purposes
(admittedly I have some trust issues with MS systems and Internet
access). With a side benefit that the AD does not supply DNS answers
directly to the clients - it's DNS workload is very low, as it is a
server with little to no need to resolve outside its authoritative
domain (just the occasional update, new software download, etc.).
Also, and maybe a bit unexpectedly the clients still update their DNS
entries (although as I mentioned earlier I don't find this all that
necessary) on the AD as even though their resolvers point to Unbound
the SRV records (cached by Unbound for all clients) are what allow
them to locate the AD and update.
At any rate the performance is quite good, and (if anecdotal
"evidence" can be offered) improved (I ran no performance tests). But
more importantly I have piece of mind that the AD has very limited
Internet exposure.
I have also run with all clients pointing to the AD with a forward to
the Unbound server (no stub-zone to the AD), which does limit the AD's
Internet footprint, but I like the setup using the stub-zone, reducing
the DNS workload on the AD and even possibly preventing some DNS
nonsense from internal systems.
Chris
I don't see the logical jump, here.
A DNS UPDATE client can identify the correct domain controller using the SOA MNAME. A recursive resolver can identify the correct domain controller for a zone by following a referral chain. Yes, some environments might have split DNS design decisions that turn out to make this tricky, but really that's more of a reflection of those design decisions than any downstream implementation decision.
There is surely no architectural requirement for the recursive resolver used by any particular stub resolver to run any particular software. "Only specify Microsoft DNS servers" (in the context of resolvers) might make good marketing copy if you're in the business of selling Microsoft DNS servers, but it doesn't sound like it's grounded in logic.
Joe
It's not just that the DNS is split (which Microsoft doesn't even support split zones within Active Directory enabled zones in a traditional "split" format), but rather, that updates are done in a multi-master scenario while sites may have replication intervals in the period of minutes, hours, or days, and updates are best processed by the local AD DNS servers (they are not forwarded upstream using the SOA record or anything else)
In a small environment none of this matters, but in large, multi-site, physically decentralized environments, you might really want local clients doing updates to a local AD server so that their DNS records appear immediately locally and that doesn't happen if you use the traditional "Update the SOA MNAME and wait for the changes to wander down to other servers"
Microsoft's DNS server is using a true multi-master, there's nothing particularly special about the server listed as MNAME, literally any AD DNS server can process updates locally and will ensure that changes are replicated out to appropriate partners.
I'm NOT saying it's the only option -- Just that it's Microsoft's best practice to use Microsoft DNS servers to service Microsoft Active Directory joined servers and clients, and in my experience, staying within Microsoft's best practices is usually wise unless you can articulate a reason to make a difference choice. If you have a practical reason to do something different, do it! I do a lot of non-standard stuff in Windows all the time.
(This isn't even a sales point, Microsoft DNS server is a free component that requires no additional licensing beyond the Active Directory environment itself.)
But at least in this case, I'm more interested in getting the benefits of unbound (awesome resolver performance, DNSSEC validation, pre-fetching, etc) without adding headache (using non-AD DNS for an Active Directory environment), so using Windows DNS internally and unbound for external resolution seems like an ideal configuration unless there are downsides (such as performance)
Hi Olaf,
Hi Unbound-Users,
I've did some benchmarking yesterday. I will continue later this month
(with different hardware, see below), but here are the first
results. I'm posting it here because you might have some ideas on how to
improve the benchmark-setup.
DISCLAIMER 1: the hardware used is not "Enterprise" or "Telco" level, I'm
aware of that. It is what I had available.
DISCLAIMER 2: this is a benchmark in a synthetic lab environment. It is
not how DNS resolvers work in the real Internet. This results can give
indications, but do not replace your own measurements in your own
environment using your own hardware.
Lab setup:
* one "load generating" machine running Unbound's own "perf" tool
(Thinkpad E330 4 Core, 16GB RAM, running Fedora Linux 20) , sending
1.000.000 queries to the DNS resolver (out of a zone of 10.000.000
IPv4-A-records). I've tested using more than one "load generating"
machine, but with the limited hardware of the resolver, more
load-machines did not change the measured results
* one "authoritative" DNS server, hosting a root zone containing
10.000.000 A-Records. BIND 9.9.5, Solaris 11, 8 GB. No delegation,
everything is inside one large root zone. TTL is 86400.
* one DNS resolver for testing Unbound vs. WinDNS. AMD Athlon 64 X2 Dual
Core 2.31 Ghz 1 Socket 2 Cores 256 L1 Cache 1 MB L2 Cache, 2GB RAM 533
Mhz DIMM, Ethernet Intel PRO/100S Desktop NIC
All machines were connected via a HP ProCurve 1410 16G Switch
I did 3 tests: Unbound on FreeBSD 10-RELEASE, Unbound on Windows 2012
ServerCore and Windows DNS Server on Windows 2012 ServerCore
Memory was measured by "top" on FreeBSD and by Windows 2012 taskmgr.
Each setup was tested in 2 runs. Run 1 with cold (empty cache) and Run 2
with filled cache (no DNS resolution towards the authoritative).
1 FreeBSD 10 unbound 1.4.20
Hi Olaf, Carsten, (unbound users),
I only just saw this thread and would like to add to Carsten's results;
we have a student working on measuring energy efficiency of DNS
resolvers, who has done some interesting measurements. While there is
probably not a one-to-one correspondence between performance and energy
efficiency we did see some rather strange effects when running Unbound
on Windows Server 2012 that warrant further investigation.
Let me briefly explain his measurement setup: he used an instrumented
server model that has special probes inserted that measure the energy
consumption of most system components (power supply, HDUs, CPU, memory,
...). On this machine he performed repeated experiments where he plays
back a captured set of queries from a live DNS environment to three
different DNS resolver implementations:
- Microsoft built-in DNS
- BIND
- Unbound
He does this with MS Windows Server 2012 as underlying OS and has
repeated the experiment with Centos 6.5 as underlying OS (obviously, he
could not measure MS DNS on that ). He has measured both without
DNSSEC validation enabled and with DNSSEC validation enabled.
Briefly summarised he found the following:
- MS Windows 2012 baseline power consumption is significantly (~10%)
lower than Centos 6.5 on the same hardware
- MS DNS uses the least energy to process the query data set, followed
by BIND then Unbound on Windows 2012
- On Linux, with DNSSEC disabled BIND uses the least energy followed by
Unbound, with DNSSEC enabled both use about the same amount of energy
All tests were run on a single CPU core.
There is ony important "BUT" here: the measurements for Unbound on
Windows 2012 were extremely unreliable. Where he saw a steady power
consumption coupled with a steady CPU load for MS DNS and BIND, for
Unbound the picture is really garbled. Lots of small peaks in CPU load
(and conversely power consumption) made the measurements extremely
noisy. Unfortunately, he did not have time to get to the bottom of this,
but it seems to suggest that there is some underlying problem here. We
are hypothesising issues were Unbound clashes with the Windows scheduler
in some way, but haven't gotten any further than hypotheses. I'm unsure
whether Unbound is compiled to use libevent on Windows, if that is the
case then that may be worthwhile investigating as well.
Note that on CentOS we do not see this behaviour.
Bottom line: Carsten's results seem to point to Unbound performing
reasonably well on MS Windows 2012, but our student's observations seem
to suggest that something weird is happening when running Unbound on
Windows Server that may be worth investigating.
I plan to find some time to visit NLnet Labs with these results at some
point in the near future, if anybody from labs is planning on attending
the DNSSEC Workshop at ICANN next week give me a shout off list and we
can have a short discussion there.
Hope this helps!
Cheers,
Roland