Adding root servers as local secondary zone to local caching server

Unbound
1

Hello,

Ok, so, I maintain a smallish Windows AD Domain, and was doing some
research on the question of whether or not to use DNS forwarders in the
Microsoft DNS server, or to use just the root hints.

I ran across an article written by someone who suggested something I'd
never considered - he suggested deleting the root hints, and setting up
new forward/reverse secondary zones that basically act as a local 'root
hints', and was wondering if anyone here had ever considered this, done
it, or has good arguments against doing it.

Here is the Microsoft Technet article:
https://social.technet.microsoft.com/Forums/ie/en-US/2f35cae2-341c-4bfe-9dac-724ddace6d51/dns-question-root-hints-vs-forwarders?forum=winserverNIS

And here is his post in response to someone who was arguing for always
just using root hints:

Well... if you're focusing on availability and speed then you may
also push the setup a bit farther, what I mean is the following:>
open your DNS management console

delete (yes, delete) the root hints

go to "forward zones", right click, create a new forward zone, type
secondary standard, name "." (a single dot)and enter the following
IPs for the authoritative DNS

192.112.36.4
192.33.4.12
192.5.5.241
193.0.14.129

confirm and move to "reverse zones", again, create a new reverse
zone, type secondary standard, name "arpa" and enter the following
IPs as the auth DNS

192.112.36.4
192.33.4.12
192.5.5.241
193.0.14.129

wait a bit for the zone transfers to take place and then have a look
at the zones, using such a config your DNS will basically act as a
"slave root" DNS that is, will keep a copy of the forward and
reverse root zones

So... will this actually work as it appears, and if so, is it a good idea?

Also - is it possible to do the same thing in Unbound, and if so, how?

Thanks...

Charles

2

Sure it is possible!

What you are asking is AXFR transfer of the root zone file. A select
number of the root servers support AXFR zone transfer with your local
DNS server setup with a read-only copy (stub node).

https://github.com/NLnetLabs/unbound/blob/master/doc/example.conf.in#L1002-L1031
https://www.isc.org/docs/Apricot2017.pdf
https://datatracker.ietf.org/doc/html/rfc7706#appendix-B
https://datatracker.ietf.org/doc/html/rfc8806#appendix-B

Another way to go about it is by downloading the root.hints file from
internic.net using a local cron job instead of AXFR zone transfer.

https://www.internic.net/domain/named.cache
https://wiki.archlinux.org/title/Unbound#Root_hints

3

Thanks Chriztoffer!

Sorry, got busy and just now coming back to this…

So, one thing you didn’t answer was - is this even a good idea?

It sounds great in theory, but sometimes reality works out very differently.

I’m specifically wondering about performance and resource usage.

E.g., would Unbound need the entire zone to be fully loaded into RAM? If so, how much RAM would be needed?

Thanks again for the links for how to do this!

Charles

4

It appears that Charles Sharp via Unbound-users <charles@cocosolutions.com> said:

-=-=-=-=-=-

Thanks Chriztoffer!

Sorry, got busy and just now coming back to this...

So, one thing you didn't answer was - is this even a good idea?

It sounds great in theory, but sometimes reality works out very differently.

I'm specifically wondering about performance and resource usage.

E.g., would Unbound need the entire zone to be fully loaded into RAM? If
so, how much RAM would be needed?

It works great. The root zone is not very big. It's under 22,000
records including all of the DNSSEC signatures, master file is about 2
meg. Also, all modern computers have virtual memory so "fully loaded
into RAM" doesn't mean anything.

On my not very busy FreeBSD server with the root, arpa, in-addr.arpa,
and ip6.arpa zones all loaded into unbound, the virtual size is 43MB,
resident size 20MB. On computers with gigabytes of RAM, those numbers
are insignificant.

5

Zones fetched using AXFR is cached locally on disk by your resolver daemon.

Your local resolver daemon will regularly check in with the upstream
server to monitor if the serial number has changed. If the upstream
number is higher than the locally cached one. Request a new copy of
the upstream zone file.

Unless you are fetching VERY large zone files over the internet (e.g.
multiple hundreds of megabytes and above). Performance concerns are
generally not something you will ever need to worry about.

If you want to run your local resolver without relying on upstream
forwarders doing the lookups for you (think e.g. 1.1.1.1 (CloudFlare),
8.8.8.8 (Google), 9.9.9.9 (Quad9)) configuring your local resolver to
use a cached root.hints file fetched using HTTP/HTTPS/AXFR is almost
your only option.
Caching the root.hints file locally *saves* your local resolver from
needing to first contact a root server for a lookup of which NS is
responsible e.g. the .NL-zone.
Without a cached root.hints file, your local resolver *needs* to
contact a root server for info of which NS's are responsible for e.g.
the .NL-zone.

Best, Chriztoffer

6

Hi Chriztoffer,

Zones fetched using AXFR is cached locally on disk by your resolver daemon.

Your local resolver daemon will regularly check in with the upstream
server to monitor if the serial number has changed. If the upstream
number is higher than the locally cached one. Request a new copy of
the upstream zone file.

Both of these aspects are known to suffer from operational problems from time to time.

Filesystems can fill up, permissions can be changed accidentally, and other things can cause files not to be able to written to disk in a way that is not always obvious to notice, since DNS resolver software is often capable of running just fine without write access to filesystems.

The ability to perform a zone transfer from a particular destination can also fail in ways that will not be obvious for long periods of time. There are no sources of the root zone by AXFR that are specified by the IETF and those that are available are often attached to anycast addresses that are not the best architectural choice for stateful transactions. The use of other protocols to transfer the root zone is also somewhat operationally novel.

I am not trying to cast doubt on the ability to retrieve a root zone and use it in this way, but I think it's important to mention that a failure to transfer can sometimes be difficult to notice, and can have operational effects that are often delayed (e.g. an RRSIG expiring days after some package was updated or some firewall was modified; a delegation change that might reduce the effective breadth of the NS set for some top-level domain; a newly-delegated and hence lightly-used TLD that works elsewhere but not through your resolver, etc).

If you want to run your local resolver without relying on upstream
forwarders doing the lookups for you (think e.g. 1.1.1.1 (CloudFlare),
8.8.8.8 (Google), 9.9.9.9 (Quad9)) configuring your local resolver to
use a cached root.hints file fetched using HTTP/HTTPS/AXFR is almost
your only option.

Note that this is not what you are doing. The root hints allow a resolver to execute a priming query. This is something that every resolver needs to do, regardless of whether it is configured to serve a local copy of the root zone.

Caching the root.hints file locally *saves* your local resolver from
needing to first contact a root server for a lookup of which NS is
responsible e.g. the .NL-zone.

Note that in many cases this is an insignificant optimisation. The round-trip penalty to obtain a referral to the NL servers (your example) is often amortised over many billions of subsequent queries and quickly trends towards zero. The ability to avoid looking up names which result in NXDOMAIN responses from the root zone can be optimised using aggressive NSEC caching in a way that avoids the operational risks alluded to above.

To me, even though the risks of failure through caching a local copy of the root zone are low, the benefits are lower, and it is not something I would choose to do. This risk assessment belongs in the hands of the operator, though, and it's certainly legitimate for other people to find a different balance.

Joe

7

Both of these aspects are known to suffer from operational problems from time to time.

Filesystems can fill up, permissions can be changed accidentally, and other things can cause files not to be able to written to disk in a way that is not always obvious to notice, since DNS resolver software is often capable of running just fine without write access to filesystems.

Agreed. Monitoring is always essential.

The ability to perform a zone transfer from a particular destination can also fail in ways that will not be obvious for long periods of time. There are no sources of the root zone by AXFR that are specified by the IETF and those that are available are often attached to anycast addresses that are not the best architectural choice for stateful transactions. The use of other protocols to transfer the root zone is also somewhat operationally novel.

The availability is there, and never likely to get included in a BCP /
RFC document. (Maybe into a draft, thou not likely a draft adopted by
a WG?)

Note that in many cases this is an insignificant optimisation. The round-trip penalty to obtain a referral to the NL servers (your example) is often amortised over many billions of subsequent queries and quickly trends towards zero. The ability to avoid looking up names which result in NXDOMAIN responses from the root zone can be optimised using aggressive NSEC caching in a way that avoids the operational risks alluded to above.

To me, even though the risks of failure through caching a local copy of the root zone are low, the benefits are lower, and it is not something I would choose to do. This risk assessment belongs in the hands of the operator, though, and it's certainly legitimate for other people to find a different balance.

It the operators choice, indeed.

If ones resolver is recently well connected to well-connected
upstreams/transit/ixp's. The benefits are indeed very very low. And
the benefits are negligible down to non-existent.

If one is hanging on a single upstream which is not well connected or
ones upstreams are of varying consistency/quality resulting in poor
user experience due to "significant" load time consumed by DNS
lookups. "Some" improvement could be gained by caching zone
information locally.
Thou, likely a DNS resolver which does speculative lookups on the most
popular DNS names ahead of the resolvers internal cache entry expering
might be a better choice. [0]

[0]: https://knot-resolver.readthedocs.io/en/stable/modules-predict.html

8

Thanks to all who responded, just the kind of info I was interested in…

So, sounds like, while it will work, and as long as reasonable precautions are taken (that I would always take anyway, like making sure the FS doesn’t fill up, that would be a total noob mistake), it should be fine, the benefits are minuscule for any system that has a decent connection.

So, one last question. I know this may just be a personal preference, but…

Do most of you use the root hints or forwarders?

I currently use the following, in order:

1.1.1.1
9.9.9.9
8.8.8.8

Thanks again!

Charles

9

Do most of you use the root hints or forwarders?

Tried both. Now using forwarders.

I currently use the following, in order:

1.1.1.1
9.9.9.9
8.8.8.8

No IPv6? Or not on a dual-stacked endpoint?

Joke questions aside, my own upstream is reasonably well-connected.
Has had configured the local dns forwarder to use root hints in the
past. Compared to using the "big cdn forwarders" the user experience
*perceived* is 99 % non-existent if the avg. lookup time is low.

If configuring the local resolver to cache all lookups for a reasonbly
minimum amount of time (e.g. 5 - 60 min). Only the user doing the
lookup when the cache entry is cold will (maybe) notice a delay if the
lookup is "slow".

Side-note: Depending on your choice of local resolver software. Some
implementations will ask the configured forwarders one at a time (i.e.
try entry 1, try 2 if 1 fail, try 3 if 2 fail, etc.).
Others (e.g. dnsmasq) defaults to ask _all_ configured forwarders
simultaniously.
Others will "regularly" test all configured forwarders to measure the
response time and only use the fastest forwarder.

10

This is an interesting thread. If I understand it correctly, with the following changes to unbound.conf
​
​auth-zone:
`

name: “.”
primary: 198.41.0.4 # a.root-servers.net
primary: 199.9.14.201 # b.root-servers.net
primary: 192.33.4.12 # c.root-servers.net
primary: 199.7.91.13 # d.root-servers.net
primary: 192.203.230.10 # e.root-servers.net
primary: 192.5.5.241 # f.root-servers.net
primary: 192.112.36.4 # g.root-servers.net
primary: 198.97.190.53 # h.root-servers.net
primary: 192.36.148.17 # i.root-servers.net
primary: 192.58.128.30 # j.root-servers.net
primary: 193.0.14.129 # k.root-servers.net
primary: 199.7.83.42 # l.root-servers.net
primary: 202.12.27.33 # m.root-servers.net
primary: 192.0.47.132 # xfr.cjr.dns.icann.org
primary: 192.0.32.132 # xfr.lax.dns.icann.org
fallback-enabled: yes
for-downstream: no
for-upstream: yes

`
​

and commenting out the root-hints, Unbound can cache the root servers in memory and perform AXFR transfer for the root zone.
​
​#root-hints: “/opt/var/lib/unbound/root.hints”

11

Do most of you use the root hints or forwarders?

Tried both. Now using forwarders.

I currently use the following, in order:

1.1.1.1
9.9.9.9
8.8.8.8

<snip/>

Side-note: Depending on your choice of local resolver software. Some
implementations will ask the configured forwarders one at a time (i.e.
try entry 1, try 2 if 1 fail, try 3 if 2 fail, etc.).
Others (e.g. dnsmasq) defaults to ask _all_ configured forwarders
simultaniously.
Others will "regularly" test all configured forwarders to measure the
response time and only use the fastest forwarder.

Unbound uses the last described behavior, but by default randomly selects one of the upstream name servers (forwarders) that falls within a 400 msec window, see also https://www.nlnetlabs.nl/documentation/unbound/info-timeout/.

There are options to prefer the faster over the slower upstream/forwarder within the time window, but eventually all of them are probed to keep the RTT updated. (See the unbound.conf fast-server-permil and fast-server-num options.)

-- Benno

12

Mark Abram via Unbound-users writes:

> This is an interesting thread.

Yes, but what I don't understand is the part in this discussion
about the root hints file. Is this just a confusion is the terminology?
Anyway, let me clarify what the root hints are and how they are
used.

As John Levine already states, the root hints play only a rile
during the priming (boots trapping) phase when the resolver starts.
For a full formal description of this process, see RFC 8109
"Initializing a DNS Resolver with Priming Queries"
(<https://datatracker.ietf.org/doc/rfc8109/&gt;\).

A somewhat easier to read explanation one can find in the isc.org
article titled "Root hints - a collection of operational and
configuration FAQs" (<https://kb.isc.org/docs/aa-01309&gt;\). This
describes this for bind of course, but the principle is the same
foe most recursive DNS resolvers.

A very short description is the section on "Root Hints" on the IANA
site (<https://www.iana.org/domains/root/files&gt;\).

Just as most other resolver, these IANA root hints are actually
always used by unbound since they are compiled in. Only when one
wants to use a completely different root there is a need to supply
a separate root hints file.

Hope this helps.

  jaap

13

Thanks Jaap, but I do have a follow-up since no one responded to my last question about root hints vs forwarders.

My understanding was that you can use either/or.

Meaning, if you use root hints, you don’t need to use forwarders, but more importantly, if you use forwarders, then the root hints are not used.

So… is this true? Or can you expound on this?

Thanks…

14

Charles Sharp via Unbound-users writes:

> Thanks Jaap, but I do have a follow-up since no one responded to my last
> question about root hints vs forwarders.
>
> My understanding was that you can use either/or.

I'm afraid you seem to misunderstand. You don't need to choose
between root hints or forwarders. Root hints are always used for
bootstrapping.

>
> Meaning, if you use root hints, you don't need to use forwarders, but
> more importantly, if you use forwarders, then the root hints _*are not
> used*_.

The root hints (either supplied or specified in the configuration
file) don't play a role while resolving a name. They are solely
used during bootstrapping the server.

Forwarders are actually not needed. Remember that Unbound is a
validating, recursive, caching DNS resolver. So if you send a query
to a unbound, it will try to resolve (recursively) the name, provide
the answer and stores the answer in the cache so it can answer
directly from the cache the next time it is queried for the same
information.

Forwarder clauses specifies that unbound doesn't need to do the
resolving part itself but to just ask another server. So it "forwards"
the query. It then returns received information and stores that in
the cache for possible later use.

Let me cite the the (more precise) description from the
unbound-configuration manual:

Forward Zone Options
       There may be multiple forward-zone: clauses. Each with a name: and zero
       or more hostnames or IP addresses. For the forward zone this list of
       nameservers is used to forward the queries to. The servers listed as
       forward-host: and forward-addr: have to handle further recursion for
       the query. Thus, those servers are not authority servers, but are
       (just like unbound is) recursive servers too; unbound does not perform
       recursion itself for the forward zone, it lets the remote server do it.
       Class IN is assumed. CNAMEs are chased by unbound itself, asking the
       remote server for every name in the indirection chain, to protect the
       local cache from illegal indirect referenced items. A forward-zone en-
       try with name "." and a forward-addr target will forward all queries to
       that other server (unless it can answer from the cache).

So note that using forwarders is a choice by the operator of the
unbound server. It is not required for running unbound.

> So... is this true? Or can you expound on this?

I hope my explanation helps.

Regards,

  jaap