DNS versus NAT ?

Unbound
1

Greetings,

I have a rather simple question about using DNS in conjunction with NAT,
as implemented on and by a typical SOHO router. I confess that my question
isn't about Unbound specifically, but about the flow of DNS queries and
responses in a NAT'd environment generally. Regardless of that, I hope
no one will begrudge me too much for asking my question here.

Imagine the following simple scenario... A particular home network consists
of two UNIX and/or Linux machines, both of which are configured to speak
to a perfectly ordinary, garden variety off-the-shelf modern SOHO router.
Each of these two UNIX/Linux machines is configured to use a unique but
statically assigned RFC1918 IPv4 address when speaking to the router...
let's just say 192.168.7.2 and 192.168.7.3. Meanwhile, the router itself
has its WAN port configured to obtain an IP address dynamically, via DHCP,
from the relevant ISP. (Not that that matters at all to my question, but
I include this detail for the sake of completeness.)

My question is just this: In such a scenario, may -both- of the UNIX/Linux
systems run its own name server, e.g. their own separate instances of, say,
unbound? And if so, what is the exact mechanism by which DNS -responses-
that arrive at the router's incoming WAN port get parceled out properly
and directed, properly, on to the specific one of the two UNIX/Linux machines
that originally sent out the corresponding DNS query?

I should say that I -believe- that I have a generally adequate, even though
only rudimentary understanding of how a typical SOHO router manages to
insure that all incoming *TCP* packets seen coming in to the WAN port are
routed to their proper final destination machines on the local network.
In that case even I, in my abundant ignorance, I can easily imagine how
the SOHO router must or might keep track of all existing TCP connections,
such that all TCP response packets are sent on, by the router, to the proper
local machine.

But much or most DNS is performed via connectionless UDP datagrams, so I
am at a loss to understand or even imagine how two or more instances of,
say, Unbound... or Bind for that matter... could successfully co-exist,
on a single home network, together, behind a single typical SOHO router.

Any enlightenment would be appreciated. Thanks.

Regards,
rfg

P.S. Of course, I would also like to know if such a scenario would either
require or benefit from specific configuration of one or both of the Unbound
instances, e.g. to explicitly... and perhaps uniquely... set the port number
that each of the two Unbound instances uses for its own outbound query
transmissions.

2

…

But much or most DNS is performed via connectionless UDP datagrams, so I
am at a loss to understand or even imagine how two or more instances of,
say, Unbound… or Bind for that matter… could successfully co-exist,
on a single home network, together, behind a single typical SOHO router.

…

UDP packets have source and destination port numbers, just like TCP. So even though UDP is connectionless, the state can be tracked in the NAT table by the source-destination port numbers just like TCP.

Tom

3

In message <C8EB50CF-E021-4E32-9762-E65D0673B91A@samplonius.org>,

...

But much or most DNS is performed via connectionless UDP datagrams, so I
am at a loss to understand or even imagine how two or more instances of,
say, Unbound... or Bind for that matter... could successfully co-exist,
on a single home network, together, behind a single typical SOHO router.

...

UDP packets have source and destination port numbers, just like TCP. So
even though UDP is connectionless, the state can be tracked in the NAT
table by the source-destination port numbers just like TCP.

Thank you, but this does not answer my question.

Please allow me to restate it again.

Assume there exist two instances of, say, Unbound, running on two machines,
both behind the same single SOHO router which is doing NAT for the local
network.

If they both send outbound DNS queries at about the same time, and both
happen to select the exact same outbound port number to do so, say for
example UDP port 53, then when the two DNS response packets come back
to the NAT router, how will it know which of the two machines it should
send each of those two DNS response packets to?

For the outbound DNS query packets, does the router re-jigger the orginal
source port numbers so that they will (hopefully) not conflict and so that
the DNS response packets, when they arrive, can be directed appropriately
to one machine or the other?

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession?

(That last question might sound silly but it isn't. I do a lot of research
relating to DNS, and the scenario described is actually not at all far
fetched in my case.)

Regards,
rfg

4

Assume there exist two instances of, say, Unbound, running on two machines,
both behind the same single SOHO router which is doing NAT for the local
network.

If they both send outbound DNS queries at about the same time, and both
happen to select the exact same outbound port number to do so, say for
example UDP port 53 [...]

It's a long time since sourcing queries from a recursor only from
port 53 was best practice. Source port randomization is the
current best practice. However, of course, "collisions" in time
may stil occur.

[...], then when the two DNS response packets come back
to the NAT router, how will it know which of the two machines it should
send each of those two DNS response packets to?

That depends on how state is maintained in your NAT box. In all
likelyhood it maintains a 4-tuple, including both the source and
destination (address and port), so if the queries use the same
source port but queries different external name servers, the NAT
box would still be able to forward correctly.

For the outbound DNS query packets, does the router re-jigger the orginal
source port numbers so that they will (hopefully) not conflict and so that
the DNS response packets, when they arrive, can be directed appropriately
to one machine or the other?

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession?

That all depends on your SOHO router, and isn't so much about DNS
per se. I can however imagine that it's quite possible to put
the SOHO router under strain, not just by using lots of queries
(using different source ports) in rapid succession, but also by
sending them to a lot of different external name servers.

Regards,

- Håvard

5

For the outbound DNS query packets, does the router re-jigger the orginal
source port numbers so that they will (hopefully) not conflict and so that
the DNS response packets, when they arrive, can be directed appropriately
to one machine or the other?

Yes. The long version is RFC 4787.

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession?

Almost certainly :slight_smile: Even quite big NAT boxes will get indigestion if you
put a lot of DNS traffic through them. In general it's best to keep
stateful middleboxes away from DNS servers. In your case you are probably
better off either setting up a DMZ at home (if they will give you multiple
IP addresses) or get a colo box for high volume DNS query traffic.

Tony.

6

In article <alpine.DEB.2.20.1906130953120.16652@grey.csi.cam.ac.uk> you write:

7

In message <20190613.084613.1255667513423551263.he@uninett.no>,

It's a long time since sourcing queries from a recursor only from
port 53 was best practice. Source port randomization is the
current best practice.

Yes, I am aware of that. I just posited a simple case that would make
the issue/question entirely apparent.

However, of course, "collisions" in time may stil occur.

Exactly so. Even with port randomization, there's nothing to stop two
machines that happen to be behind the same NAT router from both electing
to send out DNS queries on, say, port 12345 at about the same time.

[...], then when the two DNS response packets come back
to the NAT router, how will it know which of the two machines it should
send each of those two DNS response packets to?

That depends on how state is maintained in your NAT box. In all
likelyhood it maintains a 4-tuple, including both the source and
destination (address and port), so if the queries use the same
source port but queries different external name servers, the NAT
box would still be able to forward correctly.

I understand this notion of the 4-tuple and how it would or could be used
to disambiguate in this context. What I am entirely less sure about is
whether or not common off-the-shelf inexpensive SOHO routers create and
maintain one such 4-tuple for essentially each and every outbound UDP
packet they process, regardless of type. Do they? Or do they perform
some limited form of deep packet inspection so that they can create and
maintain one such 4-tuple only and exlusively for DNS query packets, in
particular?

Of course, all this makes me curious about other relevant capabilities
and limitations of SOHO routers too. Once created, how long would such
a router be likely to maintain such a (tracking) 4-tuple before discarding
it as no longer needed? And of course there is that other capacity
question that I asked:

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession?

That all depends on your SOHO router, and isn't so much about DNS
per se. I can however imagine that it's quite possible to put
the SOHO router under strain, not just by using lots of queries
(using different source ports) in rapid succession, but also by
sending them to a lot of different external name servers.

Yes. I see. Because that also would lead to the creation of multiple/
numerous 4-tuples that would all have to be strored and maintained.

I think that I have opened a can of worms for myself by either asking about
or even thinking about any of this. :slight_smile: But it has been enlightening,
and I thank you for your answer Havard. Going forward, and in light of
these issues, I most certainly -won't- be doing any my DNS research from
behind my little SOHO router, based on what I know now. I also am looking
at my SOHO router with a much more jaundiced eye now, and wondering how many
of these kinds of boxes have been subjected to serious testing for possible
programmed UDP-based Denial of Service attacks, either from the WAN side
or from the LAN side.

Regards,
rfg

8

In message <alpine.DEB.2.20.1906130953120.16652@grey.csi.cam.ac.uk>,

For the outbound DNS query packets, does the router re-jigger the orginal
source port numbers so that they will (hopefully) not conflict and so that
the DNS response packets, when they arrive, can be directed appropriately
to one machine or the other?

Yes. The long version is RFC 4787.

Thank you. I am and will be reading that.

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession?

Almost certainly :slight_smile: Even quite big NAT boxes will get indigestion if you
put a lot of DNS traffic through them.

I didn't know that. So I learned something today.

In general it's best to keep
stateful middleboxes away from DNS servers. In your case you are probably
better off either setting up a DMZ at home (if they will give you multiple
IP addresses) or get a colo box for high volume DNS query traffic.

Yes. Thank you. The latter is already in progress.

(I am assuming ... perhaps incorrectly... that a -dedicated- box will not
be absolutely necessary, as long as I have a dedicated and non-dynamic IP
address with it.)

Regards,
rfg

9

In article <1421.1560564619@segfault.tristatelogic.com> you write:

Yes. Thank you. The latter is already in progress.

(I am assuming ... perhaps incorrectly... that a -dedicated- box will not
be absolutely necessary, as long as I have a dedicated and non-dynamic IP
address with it.)

Dedicated box? Naah. Like I said, a $4/mo VPS should do it. It's a
virtual machine with a dedicated real IP address. DNS queries and
responses are usually pretty small so even a lot of DNS traffic is not
a lot of megabytes.

10

In message <20190614024738.C5395201547D1D@ary.qy>,

In article <alpine.DEB.2.20.1906130953120.16652@grey.csi.cam.ac.uk> you write:

And if that is the case, then will my SOHO router catch fire if and when
I elect to send out through it a set of 65536 or more separate DNS queries,
all in rapid succession? ...

IP addresses) or get a colo box for high volume DNS query traffic.

This sounds like a job for a $5/mo linux VPS at any of a zillion
hosting companies. It gets its own static public IP address, no
NAT nonsense needed.

I am in agreement. I had been doing my DNS research from a static IP
associated with an end luser broadband line, but I am re-jiggering
my entire network now and plan to get rid of -that- static IP. And
that's what prompted my question(s). Apparently, once I make this
change, I won't be able to just carry along as I had been doing before.
Instead, as it now seems, I'll have to move my DNS research to some
cloudish sort of place.

11

You may not need a "cloudish sort of place." It really depends your user count. A residence or small business doesn't generate that many "new" domain queries in 24 hours.

If you configure Unbound cache parameters correctly, it won't need to query much. One trick is setting minimum TTL 5-15 minutes to prevent misapplication of TTL for load balancing. Even for a guest-consumer network like a cafe or small hotel, its google, facebook, instagram, twitter, and (repeat). If you install an adblock/malweb list in causes 'local-domain: bad.example.com static', then you can really cut down on useless DNS. The same excessive video-scripted ads that bog your browser also bog DNS. I package Unbound for OpenWrt and it works okay on single core mips (Tplink Archer C7) and it works well on dual core ARM (Linksys WRT3200ACM). In this case, it also works to serve the router as the only DNS provider by announcing it in DHCP and RA, and also firewall all UDP/TCP 53 trying to cross WAN/LAN. Only allow the router to be a target in the firewall.

The "cloudish" option can also be DNS-over-TLS to cloudflare 1.1.1.1 or quad9 9.9.9.9. Then Unbound merely forwards the full query and these providers do all the heavy lifting. These services appear to have reasonable privacy policies at least worth reading. With TLS, your ISP cannot mingle some "extra information" into your DNS responses.

12

In article <8edb08ac-5f86-04b7-7b7e-8bf1eb25386c@gmail.com> you write:

You may not need a "cloudish sort of place." It really depends your user
count. A residence or small business doesn't generate that many "new"
domain queries in 24 hours.

I'm pretty sure that when Ron said 64K outstanding queries, he meant
it. It's not just family members looking at Facebook.

The "cloudish" option can also be DNS-over-TLS to cloudflare 1.1.1.1 or
quad9 9.9.9.9. Then Unbound merely forwards the full query and these
providers do all the heavy lifting. These services appear to have
reasonable privacy policies at least worth reading. With TLS, your ISP
cannot mingle some "extra information" into your DNS responses.

That's not a bad idea, if the performance is adequate and it can deal
with all those intermingled queries on a few DoT connections.

R's,
John

13

In message <20190615154602.3BD08201591C0B@ary.qy>,

In article <8edb08ac-5f86-04b7-7b7e-8bf1eb25386c@gmail.com> you write:

You may not need a "cloudish sort of place." It really depends your user
count. A residence or small business doesn't generate that many "new"
domain queries in 24 hours.

I'm pretty sure that when Ron said 64K outstanding queries, he meant
it. It's not just family members looking at Facebook.

Well, to be clear, I never said 64+K queries all "outstanding" (and as
yet unanswered) at any given moment. In fact, my hope and believe is
that my worst case for simultaneously open/pending queries would likely
be smaller than that. However I have been known to do a million or
so DNS queries in an afternoon, and depending on how the SOHO router
maintains it's table of connection-ish 4-tuples, doing that from behind
some such router might indeed cause the thing to catch fire, metaphorically
speaking.

A lot of this depends on one's defintition of an "outstanding" DNS query
also. If I do a million queries, to all sorts of things scattered all
over the place... which is something that I do routinely... then it's very
typical that as much as a quarter or more of thoes DNS queries will go
entirely unanswered due to dead delegations. So if I send out 1 million
queries over the space of, say, 3 hours, at the end of those 3 hours we
mighy say that 250,000 queries are still "outstanding" because no response
whatsoever has been received. So obviously, if the router is going to
cling onto and keep each 4-tuple that is associated with each of those, for
hours on end, and not do garbage collection early and often, then that's
going to be a problem.

To bring this back, at least vaguely, to being on topic, what is Unbound's
approach to this problem? Has anyone tried to shove a few gazllion queries
through it over a very short period of time, just to see if it could be
made to explode? If not, doing so might be entertaining.

(Memories of various videos I've seen which involve the combination of
Mentos and Diet Coke are springing immediately to mind. :slight_smile:

Regards,
rfg

14
15

In message <624735c3-296a-f0d7-519c-3c75327445e2@wagsky.com>,

Many SOHO all-in-one routers are running Linux kernels.

Yes, I believe that mine is.

Of course, if you're really running that kind of DNS volume
and the TCP traffic that would usually go with it, you're
probably not using a SOHO-grade SoC for your router with
only 128 MB of RAM.

Well, I've been doing this stuff, so far, which is to say until now,
from a direct connected DSL line.... NOT from behind my SOHO router.

I had planned to change that, but now I see that doing so would be,
um, suboptimal.

Regards,
rfg

16

My DANE/DNSSEC survey machine (Low-power 25W Xeon SuperMicro) is
also my home router. It performs NAT for the inside network, but
DNS traffic is specifically excluded from NAT early in the firewall
rules. The internal machines must use the DNS cache on the border
machine.

With that in place, my unbound server is able to process ~2400 qps,
without running into any NAT state barriers. Without the bypass
rules, I'd overflow the NAT state table in under a minute. But
with the rules, ~9.8 million signed domains (multiple queries per
domain, covering DS/DNSKEY/MX and modulo dedup also A/AAAA/TLSA per
MX host) are processed in ~5 hours.

This yields a rather unusual unbound "cache", e.g. since last stats
reset:

    total.num.queries=21993259
    total.num.cachehits=9160

Or a cache hit rate that is less than 0.05% (under the driver blood
alcohol limit in Melbourne Australia :-).

17

Correction, I happened to tune up my client code a bit over the
last couple of days, and today a rescan of ~56 million previously
unsigned domains looking for new signatures is clocking in at 4050
qps.

The ~2400 qps number was observed while scanning already known signed
domains, where more database processing happens for each one slowing
things a bit on the client side.

Bottom line, with no NAT in the way, unbound easily manages 4k qps
on comparatively modest hardware.

top(1) shows:

  PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
  45420 viktor 8 35 0 418M 362M uwait 7 115:09 123.71% danescan
  25483 unbound 2 52 0 964M 678M kqread 3 31.1H 81.28% unbound
  45426 postgres 1 52 0 16G 14G select 1 62:59 67.29% postgres
  45428 postgres 1 52 0 16G 14G select 2 63:00 67.22% postgres
  45427 postgres 1 52 0 16G 14G select 4 62:59 67.20% postgres

Which generates 3.4 Mbps of outbound DNS query traffic and 27.3
Mbps of inbound replies.

18

Or a cache hit rate that is less than 0.05% (under the driver
blood alcohol limit in Melbourne Australia :-).

You owe me a new keyboard. I was having my coffee while reading email,
since the DNS software lists tend to be safe for this. But "under the
legal limit" is a metric I've not used for cache hit rate before. Bravo.
:slight_smile: