Multi-threaded operation?

Unbound
1

Hi,

I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that
despite me having configured

server:
  num-threads: 12

only one of the threads is handling all the queries, according to
the output from "unbound-control stats". "Not what I wanted."

I've also configured

  so-reuseport: yes

as the comment in the template config file just preceding that is

  # use SO_REUSEPORT to distribute queries over threads.

Any ideas about where I should start looking for what's amiss?

Regards,

- Håvard

2

Hi,

it looks like I'll have to answer my own question, which is a
little disappointing:

I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that
despite me having configured

server:
  num-threads: 12
  so-reuseport: yes

only one of the threads is handling all the queries, according to
the output from "unbound-control stats". "Not what I wanted."

It turns out that using the "so-reuseport" setting to distribute
the load over the threads is a fairly recent Linuxism, and
relying on it causing the kernel to distribute the load over the
different sockets is not portable.

The first answer in

  http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t

says it quite clearly:

    Linux 3.9 added the option SO_REUSEPORT to Linux as well.
    [...] Additionally the kernel performs some "special magic"
    for SO_REUSEPORT sockets that isn't found in any other
    operating system so far: For UDP sockets, it tries to
    distribute datagrams evenly, for TCP listening sockets, it
    tries to distribute incoming connect requests (those accepted
    by calling accept()) evenly across all the sockets that share
    the same address and port combination. [...]

I'll try to turn off the so-reuseport option later today, and see
if that improves the situation.

Regards,

- Håvard

3

Hi Havard,

Hi,

it looks like I'll have to answer my own question, which is a
little disappointing:

I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that
despite me having configured

server: num-threads: 12 so-reuseport: yes

only one of the threads is handling all the queries, according
to the output from "unbound-control stats". "Not what I
wanted."

It turns out that using the "so-reuseport" setting to distribute
the load over the threads is a fairly recent Linuxism, and relying
on it causing the kernel to distribute the load over the different
sockets is not portable.

Yes, I did not realise this. If you disable reuseport on BSD I think
it will start to work.

The first answer in

http://stackoverflow.com/questions/14388706/socket-options-so-reuseadd

r-and-so-reuseport-how-do-they-differ-do-they-mean-t

says it quite clearly:

Linux 3.9 added the option SO_REUSEPORT to Linux as well. [...]
Additionally the kernel performs some "special magic" for
SO_REUSEPORT sockets that isn't found in any other operating system
so far: For UDP sockets, it tries to distribute datagrams evenly,
for TCP listening sockets, it tries to distribute incoming connect
requests (those accepted by calling accept()) evenly across all the
sockets that share the same address and port combination. [...]

I'll try to turn off the so-reuseport option later today, and see
if that improves the situation.

The option provides nice benefits on Linux; somehow having one file
descriptor per thread/process that is tied to the same socket allows
for much faster operation in the network stack.

Best regards, Wouter