Some performance tests

Unbound
1

Hi all,
i like to make test to try new product, so i did some tests WITHOUT any goal to demonstate which software is better.
i want compare Bind9 vs Unbound in cache-only deploy (i use it for speedup antispam server).
My server is:

quad core Intel(R) Xeon(TM) CPU 3.40GHz
4Gb Ram
LSI Logic 1020/1030 Ultra320 SCSI Adapter
Debian Etch
30Mb full internet access (!)

i setup only one forwarder (dns of biggest ISP in italy) and i used dnsperf (the same used by ISC),
i tried only 1000 queries to avoid to be considered an attacker.

These are results of three tests (mean):

unbound
  Queries per second: 215.951160 qps

bind
  Queries per second: 181.861585 qps

in this case unbound is 20% faster.

Now, i want to make a different tests with more 'sense',
Can we discuss a battery of test and a 'modus operandi' ?
in particoular which dns server must be used to obtain 'good' results (to avoid to be a bottleneck)?

Thanks,
Mauro Rappa

2

Mauro Rappa wrote:

These are results of three tests (mean):

unbound
  Queries per second: 215.951160 qps

bind
  Queries per second: 181.861585 qps

in this case unbound is 20% faster.

What OS?

3

Debian 4!

4

* Mauro Rappa:

Debian 4!

With the 2.6.18 kernel, or the 2.6.24 kernel (dubbed "etchnhalf")?

5

Hi Mauro,

These numbers are very low, both unbound and bind. I often see more than
1000 qps on very low hardware. With 1000 queries in total maybe you are
testing something else, maybe the speed of the ISP in italy?

unbound comes with a 'perf' tool (make perf), it cannot limit itself to
1000 queries, but to a number of seconds. You would be considered a
flooding maniac during those seconds. It has very few options.

I think the 1000 queries limit is somehow changing the result. With you
sending 1000 queries fairly quickly, then getting a large roundtrip time
or something, and the result dominated by that roundtrip time?

On the other hand, maybe the ISP DNS server really is that slow, and
these rates are 'normal' for your setup. In that case I would suggest
to make it more realistic, so that the numbers are more like reality.

You could run your own bind/unbound/... server as forward server, so you
can send as many queries to it as you like. An not bother the italian ISP.

Best regards,
~ Wouter

Mauro Rappa wrote:

Hi all,
i like to make test to try new product, so i did some tests WITHOUT

any goal to demonstate which software is better.

i want compare Bind9 vs Unbound in cache-only deploy (i use it for

speedup antispam server).

My server is:

quad core Intel(R) Xeon(TM) CPU 3.40GHz
4Gb Ram
LSI Logic 1020/1030 Ultra320 SCSI Adapter
Debian Etch
30Mb full internet access (!)

i setup only one forwarder (dns of biggest ISP in italy) and i used

dnsperf (the same used by ISC),

i tried only 1000 queries to avoid to be considered an attacker.

These are results of three tests (mean):

unbound
  Queries per second: 215.951160 qps

bind
  Queries per second: 181.861585 qps

in this case unbound is 20% faster.

Now, i want to make a different tests with more 'sense',
Can we discuss a battery of test and a 'modus operandi' ?
in particoular which dns server must be used to obtain 'good' results

(to avoid to be a bottleneck)?

6

Thanks Wouter,
it was exacly i want to listen, maybe the DNS server is too 'lazy' (we are in italy...)

what do you mean about setting up 'forward server'? i already forward all queries to one dns..
are you speaking abount a chain of dns?
does anybody have a 'good' public dns to use for testng?

after i'll test perf tool.

Thanks,
Mauro

7

Hi Mauro,

Mauro Rappa wrote:

what do you mean about setting up 'forward server'? i already forward

all queries to one dns..

are you speaking abount a chain of dns?
does anybody have a 'good' public dns to use for testng?

I meant you set up your own server and do not use a public dns.
You would have to flush the cache on it between tests to keep things
honest. Then you can flood that server without being antisocial.

after i'll test perf tool.

sure, the perf tool itself may perform better than the other tool :slight_smile:

Best regards,
~ Wouter

8

"forwarding" will give you only a small fraction of a resolver's capability,
most prominently its cache. While that may be a good way of running in
production (as long as the forwarder's availability is sufficient),
what you test appears to be the performance of the system you forward the
queries to. To get a clearer picture of unbound's performance, you
could test it in its full resolver mode: use no forwarder at all and
let unbound do the whole work of iteratively finding and compiling
responses to incoming queries. Unless, of course, something in your
operations environment doesn't let you do this, but then it might not
serve too well as a performance test setup.

-Peter

9

Mauro,

we are also missing patterns you are testing. Do you test entirely random domain name every time or you are repeating same queries (real question is whether resolver cache is used or not). Also what is your upstream link connectivity? Your bandwith may also affect results.

Ondrej.

2008/6/3 Mauro Rappa <Mauro.Rappa@infoklix.it>:

10

Hi all,
thank for every answer.

i do some precisations on my tests:
1) after every test i restarted daemons to empty cache (is it exact?)
2) queries are total random
3) my connectivity is 30Mb on backbone

Now i want to test :
1) try the tool mentioned before
2) change forwarder server or create my own, but i have to setup several fake domain with almost 1k entry!

I know 'caching only' setup is only a piece of Unbound, but at the moment, i'd like to focus on it.
Anyone want test other configurations?

Best regards,
Mauro