Hi,
We are successfully using libldns (C) in our project to perform DNS
queries. At some point the requirements for queries per minute went
high and in order to fulfill those we had to start fork()-ing the
process to perform queries in parallel. This works well but we are
concerned now about the amount of processes run in the system.
After searching on the net I found out about libunbound and it's
ability to perform asynchronous queries. This looks awesome, but the
questions are as follows.
Given the requirement to perform up to 20-40 DNS queries to different
name servers asynchronously in a single task, how well can unbound
handle such load (with all the possible timing outs and so on)? What
would be the recommended limit of queries in this case?
And the second question, perhaps anybody has an idea, how easy is
switching from libldns to libunbound? What are the
advantages/disadvantages of doing that?
Cheers,
VL
Given the requirement to perform up to 20-40 DNS queries to different
name servers asynchronously in a single task, how well can unbound
handle such load (with all the possible timing outs and so on)? What
would be the recommended limit of queries in this case?
Seeing that the unbound DNS server uses libunbound, you can be sure this
library can handle many orders of magnitude more than 40 qps. More like
400.000 qps 
And the second question, perhaps anybody has an idea, how easy is
switching from libldns to libunbound? What are the
advantages/disadvantages of doing that?
There is some great documentation at unbound.net/documentation
You are probably interested in:
http://unbound.net/documentation/libunbound-tutorial-4.html
http://unbound.net/documentation/libunbound-tutorial-5.html
Paul
Hi Vladimir,
We are successfully using libldns (C) in our project to perform DNS
queries. At some point the requirements for queries per minute went
high and in order to fulfill those we had to start fork()-ing the
process to perform queries in parallel. This works well but we are
concerned now about the amount of processes run in the system.
After searching on the net I found out about libunbound and it's
ability to perform asynchronous queries. This looks awesome, but the
questions are as follows.
Given the requirement to perform up to 20-40 DNS queries to different
name servers asynchronously in a single task, how well can unbound
handle such load (with all the possible timing outs and so on)? What
would be the recommended limit of queries in this case?
And the second question, perhaps anybody has an idea, how easy is
switching from libldns to libunbound? What are the
advantages/disadvantages of doing that?
We currently use a tiered approach for one of our projects, where LDNS
synthesizes the local queries, whereas a local copy of Unbound does the
resolving. One thing to bear in mind, should you choose a similar
approach, is that LDNS does not perform sanity checks on responses that
it receives on a socket. This can come back to bite you if you have lots
of open queries and if one or more time out. If your local Unbound
decides to answer an open query while LDNS has already timed out, the
answer may get picked up by a freshly opened socket that happens to be
on the same port. Since LDNS does not check if the QNAME or query ID
match the query it sent, this will get accepted.
In other words: if you use a locally running Unbound and LDNS to
synthesize and send queries, make sure you check QNAME and query ID on
the returned response.
Best regards,
Roland
Hi Vladimir,
Vladimir Levijev wrote:
We are successfully using libldns (C) in our project to perform DNS
queries. At some point the requirements for queries per minute went
high and in order to fulfill those we had to start fork()-ing the
process to perform queries in parallel. This works well but we are
concerned now about the amount of processes run in the system.
After searching on the net I found out about libunbound and it's
ability to perform asynchronous queries. This looks awesome, but the
questions are as follows.
Given the requirement to perform up to 20-40 DNS queries to different
name servers asynchronously in a single task, how well can unbound
handle such load (with all the possible timing outs and so on)? What
would be the recommended limit of queries in this case?
And the second question, perhaps anybody has an idea, how easy is
switching from libldns to libunbound? What are the
advantages/disadvantages of doing that?
We currently use a tiered approach for one of our projects, where LDNS
synthesizes the local queries, whereas a local copy of Unbound does the
resolving. One thing to bear in mind, should you choose a similar
approach, is that LDNS does not perform sanity checks on responses that
it receives on a socket. This can come back to bite you if you have lots
of open queries and if one or more time out. If your local Unbound
decides to answer an open query while LDNS has already timed out, the
answer may get picked up by a freshly opened socket that happens to be
on the same port. Since LDNS does not check if the QNAME or query ID
match the query it sent, this will get accepted.
In other words: if you use a locally running Unbound and LDNS to
synthesize and send queries, make sure you check QNAME and query ID on
the returned response.
Thanks for your reply. We actually are not using the Unbound, we are
just querying the Name Servers (IN A) that serve existing domains. So
I guess that means we have nothing to worry about and libunbound would
server well for us?
If I understand you correctly, you want to query authoritative name
servers directly, rather than perform a full recursion, is that correct?
The main purpose of libunbound is to act as a resolver, but while I have
not tried this myself, it appears you can direct it to query specific
authoritative name servers for specific domains. Given what is in the
documentation [1], you could create a resolver context and tell it to
query a specific name server for a specific domain, using
ub_ctx_set_stub(...), e.g.:
ub_ctx_set_stub(&ctx, "example.com.", "2001:500:8f::53", 0);
This tells libunbound to send queries for "example.com" (and anything
below that) to 2001:500:8f::53 (a.iana-servers.net, which is
authoritative for example.com).
If you want to achieve the same goals with ldns, this is perfectly
feasible, for example by splitting your application into a lot of
threads. We managed to run tens of thousands of queries that way without
problems, as long as you heed the advice provided above (check that the
QNAME and QID in the response match the query).
I hope this is helpful.
[1] http://www.unbound.net/documentation/doxygen/unbound_8h.html
Cheers,
Roland