Logfile/verbosity and master/slave

NSD
1

Can someone tell me if nsd (3.2.0) supports logging the actual queries
it receives (regardless of the answer, such as NXDOMAIN)?

Yes, I realize that purists will tell me it slows down the DNS server,
but for a certain application, I need to know what IP address is
requesting what record lookup. I have tried to increase the
"verbosity" but still don't get individual query information. Does
anyone know if there's a patch available to do this already? Perhaps
one that enables this feature at a certain verbosity level?

Another question - Is it possible to configure nsd such that it is
both a master and a slave for the same zones? The idea behind this is
that I'd like to be able to update any of the nsd zonefiles residing
on any of the boxes, and have that update propagate to the others.
Yes, I realize I can do this (crudely and forcefully) with a cron job
that rsyncs and triggers a reload, but I was hoping there was a better
way that actually used NOTIFY or such.

Thanks in advance... I'm converting from MaraDNS due to response
bugs, after 10 years of BIND.

Regards,
Lew Payne

2

Lew Payne writes:

Yes, I realize that purists will tell me it slows down the DNS server, but for a certain application, I need to know what IP address is requesting what record lookup.

tcpdump will tell you this, and IMO using tcpdump is better than doing it in-server, since tcpdump has a nice query language and can be started and stopped independently of the server.

Arnt

3

Lew Payne wrote:

Can someone tell me if nsd (3.2.0) supports logging the actual queries
it receives (regardless of the answer, such as NXDOMAIN)?

No. Programs like dnsmon and tcpdump can do this I believe.
you could run tcpdump dst port 53 and dst host <public address of your

.

Yes, I realize that purists will tell me it slows down the DNS server,

Exact.

Another question - Is it possible to configure nsd such that it is
both a master and a slave for the same zones? The idea behind this is
that I'd like to be able to update any of the nsd zonefiles residing
on any of the boxes, and have that update propagate to the others.
Yes, I realize I can do this (crudely and forcefully) with a cron job
that rsyncs and triggers a reload, but I was hoping there was a better
way that actually used NOTIFY or such.

Yes, simply configure. I wrote NSD3 to be able to do this :slight_smile:

Set up allow-notify, request-xfr, notify and provide-xfr to all other
servers. Please use a TSIG key for security.

This will cause all servers to notify all others, and attempt to
download from them.

The only downside is the perpetual zone problem (discussed in dnsop
draft-expire) where the zone will never expire, because every server can
update its lease from any other (master). The only way to make a zone go
away would be to de-configure the zone on the servers.

Best regards,
   Wouter

4

Wouter and Arnt - Thank you for your answers to my previous questions.

So far, nsd (3.2.0) is behaving nicely, with roughly 200 zones that it
is authoritative for.

Being able to import BIND8 zone files was a big bonus, as is having a
CLI (nsdc).

Can you tell me if nsd logs erroneous or broken queries? For example;
queries for non-authoritative domains, malformed queries (common with
exploit scripts), and queries for non-existent sub-domains (within our
authoritative zone - NXDOMAIN)? These are all very useful for
security purposes, and I (used to, until I switched) use them to feed
a daemon that tracks them and takes evasive action when necessary (via
ipfw radix table entries).

Is there any chance of getting this type of logging, at a certain
verbosity setting?

Regards,
Lew Payne

5

Hi Lew,

NSD does not do that for general queries. It does give you BIND-like
statistics about this, but they only show you how many times that
occurred, nothing about the query initiator.

I am not to keen about putting this type of logging in-server and I
agree with Ant and Wouter that you should tcpdump or something to
retrieve these queries.

Kind regards,

Matthijs Mekking
NLnet Labs

Lew Payne wrote:

6

I hope you don't mind our exchange of ideas... In arguing this point,
I do not mean any harm. With that in mind, I would like to
respectfully state the following:

NSD does not do that for general queries. It does give you BIND-like
statistics about this, but they only show you how many times that
occurred, nothing about the query initiator.

I am not to keen about putting this type of logging in-server and I
agree with Ant and Wouter that you should tcpdump or something
to retrieve these queries.

The idea behind not including query logging is solid - because a busy
DNS server should not be burdened with anything else that slows its
primary goal (answering queries). However, that idea is not contrary
to logging bad, erroneous, malformed or illegal queries. These are
the exception rather than the rule. As such, they should have no
impact on normal nameserver performance.

That leaves us with "why would we want to log these?" As you know,
security is a necessity when connecting something to the net,
especially if it is to be used to serve the general public.
Identifying and "fixing" sources (IP's, netblocks, ASN's, etc) of
garbage data is an important part of security. In fact, it may weight
favorably with the argument of not burdening the nameserver with
unnecessary operations... since the "bad" data can be blocked (if
malicious) or fixed (if a remote ISP config error).

I would argue that incorporating "bad" queries into a log, at a
certain verbosity level, can only enhance the real-time diagnostics
nsd provides to the general community. Since this can be turned off,
it does not impact customers who don't care about it (or aren't aware
of its security implications).

The problem with diagnosing this with external tools (tcpdump, et-al)
is that you must run the process in promiscuous mode, and thus
generate a security concern. Also, and more important from a
performance standpoint, it must analyze and capture each incoming
packet - good and bad. That's unnecessary processing, and a task that
nsd is already performing de-facto. nsd has the capability of
identifying bad queries already - there is no added burden in tasking
it with logging them (other than logging overhead, which I have
claimed to be minimal and helpful in this regard).

I hope you will take these points into consideration, and share them
with others as appropriate.

Regards,
Lew Payne
lew.payne@gmail.com

7

I would argue that incorporating "bad" queries into a log, at a
certain verbosity level, can only enhance the real-time diagnostics

I'm agnostic as to whether or not to include logging in nsd. However,
if it is included, I hope any added code can be disabled as a compile-time
option.

The problem with diagnosing this with external tools (tcpdump, et-al)
is that you must run the process in promiscuous mode, and thus
generate a security concern. Also, and more important from a
performance standpoint, it must analyze and capture each incoming
packet - good and bad. That's unnecessary processing, and a task that
nsd is already performing de-facto. nsd has the capability of
identifying bad queries already - there is no added burden in tasking
it with logging them (other than logging overhead, which I have
claimed to be minimal and helpful in this regard).

If an nsd server is liable to typically experience loads high enough
that a concurrent packet capture would cause significant additional
performance degradation, then it's probably better to do passive
capture on a separate box rather than add aditional processing overhead
to the existing one. (This would also mitigate the promiscuous mode
risk.)

Geoff

8

:The problem with diagnosing this with external tools (tcpdump, et-al)
:is that you must run the process in promiscuous mode

tcpdump -p

9

Matthijs Mekking writes:

I am not to keen about putting this type of logging in-server and I agree with Arnt and Wouter that you should tcpdump or something to retrieve these queries.

One thing you could usefully do is mangle the nsd source code to parse queries and basically do all the input stuff, but log any and all syntax errors and not process valid queries at all. It should NEVER send any replies or any other network output.

The idea would be to have an nsd companion program which one can run on a monitor port, or feed input from a tcpdump file, to get quick and simple analysis of some kinds of problems.

Arnt

10

My experience shows TCPdump analysis is useful at smaller scale, but it
takes significant amount of resources to compute/analyze large scale
deployments with high volume traffic. I wish NSD logs query syntax errors
provided you turn on the special log level.

I think most useful thing is traffic monitoring/analysis via NSD stats.
However, NSD stats needs improvement and should match bind features at
minimum to be really practical. I am 100% sure that NSD doesn't log per zone
stats as bind does. Tests indicate the NSD stats has some flaws with zone
transfers in conjunction with status interval.

Thanks,
Srini

11

a message of 14 lines which said:

tcpdump will tell you this, and IMO using tcpdump is better than
doing it in-server,

I agree that this logging ability should *not* be added to nsd (or
that it can be disabled at compile-time) but do note there are other
tools than tcpdump, specially when you want fine-grain selections of
DNS queries/responses, as requested by the OP. (tcpdump can only
filter by IP addresses, port numbers, not by DNS content.)

I suggest:

tshark <http://www.wireshark.org/docs/man-pages/tshark.html&gt; and its
powerful DNS filter language
<http://www.wireshark.org/docs/dfref/d/dns.html&gt;

dnscap <https://www.dns-oarc.net/tools/dnscap&gt; a pain to compile but
a very useful tool

12

Examples are with dnscap <https://www.dns-oarc.net/tools/dnscap&gt;

a message of 25 lines which said:

malformed queries (common with exploit scripts),

-sr -ef

queries for non-existent sub-domains (within our authoritative zone
- NXDOMAIN)?

-sr -ex

13

a message of 54 lines which said:

The problem with diagnosing this with external tools (tcpdump,
et-al) is that you must run the process in promiscuous mode, and
thus generate a security concern.

Adding a lot of code to NSD is a more important security concern to
me.

Most people who run such capture tools do not run them on the name
server, anyway, but on a dedicated machine, with Ethernet port
mirroring to copy the queries or responses to it.

14

This has been an interesting discussion. I (as OP) originally asked
for the ability to log bad, malformed or non-zone queries because this
has been an invaluable feature for us in detecting and blocking
malicious users. It is part of an overall security scheme, which
includes specialized distributed software that communicates to
everything in our cluster (via anycast).

I am a long-time bind user, having first adopted it in early 1998.
For the past year, I have used maraDNS for performance reasons. There
were idiosyncrasies in its operation that lead me to look for another
lightweight solution last month, in anticipation of our new data
center migration. I am now running nsd, and am very satisfied with
it. The ability to parse bind-format zone files is brilliant, as is
the use of a zone compiler to store them in bdb format. The nsdc CLI
deserves praise, too.

NSD is now being used by the site with the longest (number one, in
fact) uptime in netcraft history. We are also one of the most
attacked sites on the Internet. As such, detection of anything
"abnormal" is an important part of our security protocol. I would
love to see "rejected" queries on a real-time basis, right from nsd.

Because nsd has already parsed the query and determined that it's to
be rejected, placing such code in nsd (in my opinion) makes the most
sense and offers the least amount of overhead. The alternative is to
run yet another program, and analyze the complete IP flow... which is
a waste of CPU cycles, etc. Also, not everyone is in a position to
run "yet another tool" to obtain this simple (from an nsd perspective)
information. At NAP of the Americas, we are limited in rack space...
I can't just add another box every time I need to perform another task
- nor is it the wisest deployment of resources (unless part of
CISP/PCI requirements).

Therefore, I hope you will seriously consider adding "rejected" query
logging to nsd, settable via verbosity and perhaps even as a
compile-time option.

Regards,
Lew Payne

15

Lew,

16

Shane Kerr writes:

Only the application in question can report exactly how it has classified any given query at any given time.

tcpdump can give you information, but you must always be guessing how the name server handled any given query.

No. If you see the answer, you see the answer and know exactly what nsd did. If you don't: That's why I suggested embedding nsd's parser in an error-reporting applcation.

Arnt

17

Arnt,

Shane Kerr writes:
> Only the application in question can report exactly how it has
> classified any given query at any given time.
>
> tcpdump can give you information, but you must always be guessing how
> the name server handled any given query.

No. If you see the answer, you see the answer and know exactly what nsd
did. If you don't: That's why I suggested embedding nsd's parser in an
error-reporting applcation.

Except that not all errors are due to parsing problems. Sometimes
problems are caused by temporary issues with NSD itself. Sometimes
packets are unanswered due to filters, or OS issues, and so on.

Knowing what the actual application thinks it is seeing gives you a more
complete picture.

I realize people like NSD being small, but keeping it so small that you
leave out important things for administrators is probably not a good
thing... (to be fair, BIND 9 is only just now getting decent statistics,
so obviously this is not as critical as most other features).

18

Stephane Bortzmeyer writes:

I suggest:

tshark <http://www.wireshark.org/docs/man-pages/tshark.html&gt; and its powerful DNS filter language <http://www.wireshark.org/docs/dfref/d/dns.html&gt;

dnscap <https://www.dns-oarc.net/tools/dnscap&gt; a pain to compile but a very useful tool

Very useful. Thanks.

Arnt

19

For the time being, I've settled on this to fill my needs (uses pcap library)...

http://www.adotout.com/
( DNS Flood Detector )

I can use it in "dump" mode to see the decoded queries (and source
addresses), or in daemon mode to notify me (and hence, my
security/firewalling tool) in real-time of misbehaving queries.

Regards,
Lew Payne