increasing memory usage (using rpz zones)

Unbound
1

Hi all,

I recently upgraded to unbound 1-11.0 because I needed support for rpz
zones. while the rpz zones work fine, I realized rather quickly, that
the memory unbound uses is much higher than before and *increases* over
time. when I use a rather large rpz zone it actually unbound's memory
requirements increases rather dramatically.

These are my observations:

2

Hi all,

[replying to my own post]

Apparently it is normal that unbound uses *a lot of RAM* after the
initial load of the rpz zones (point 1 below).

The problem 2, however, is still unsolved. But I was able to track down
the memory leak (point 2 below) somewhat: it only occurs if unbound is
configured to listen on more than one IP address.

I filed a bug here:
https://github.com/NLnetLabs/unbound/issues/318

Best,
Hp

3

Hi Hanspeter,

Hi all,

[replying to my own post]

Apparently it is normal that unbound uses *a lot of RAM* after the
initial load of the rpz zones (point 1 below).

Does your RPZ zone contain a lot of records with the local data RPZ
action? Due to the way the memory allocation is done here this can
result in a very memory hungry Unbound instance. We are working on a fix
for this.

The problem 2, however, is still unsolved. But I was able to track down
the memory leak (point 2 below) somewhat: it only occurs if unbound is
configured to listen on more than one IP address.

I filed a bug here:
https://github.com/NLnetLabs/unbound/issues/318

Thanks, appreciate the detailed report. My first hunch is that this is
related to the zone transfer because of the involvement of the
configured IP addresses. We'll continue working on this and communicate
about this issue on github.

-- Ralph

4

Fixed in https://github.com/NLnetLabs/unbound/pull/329

-- Ralph

5

Hi Hanspeter,

> Hi all,
>
> [replying to my own post]
>
> Apparently it is normal that unbound uses *a lot of RAM* after the
> initial load of the rpz zones (point 1 below).

Does your RPZ zone contain a lot of records with the local data RPZ
action? Due to the way the memory allocation is done here this can
result in a very memory hungry Unbound instance. We are working on a
fix
for this.

I am not entirely sure what "local data RPZ action" means. almost all
our records in the rpz zones are CNAMES.

> The problem 2, however, is still unsolved. But I was able to track
> down
> the memory leak (point 2 below) somewhat: it only occurs if unbound
> is
> configured to listen on more than one IP address.
>
> I filed a bug here:
> https://github.com/NLnetLabs/unbound/issues/318

Thanks, appreciate the detailed report. My first hunch is that this
is
related to the zone transfer because of the involvement of the
configured IP addresses. We'll continue working on this and
communicate
about this issue on github.

perfect, thanks,
hp

6

Hi Hanspeter,

Hi Hanspeter,

Hi all,

[replying to my own post]

Apparently it is normal that unbound uses *a lot of RAM* after the
initial load of the rpz zones (point 1 below).

Does your RPZ zone contain a lot of records with the local data RPZ
action? Due to the way the memory allocation is done here this can
result in a very memory hungry Unbound instance. We are working on a
fix
for this.

I am not entirely sure what "local data RPZ action" means. almost all
our records in the rpz zones are CNAMES.

(All RPZ actions use CNAME <data>)

"Local data” action means that the RPZ zone you’re supplying has an “alternative” answer that’s presented to querying client, redirecting the client to another host. (This explains why I don’t see your unbound's memory-hogging behaviour on SUNET unbound instances.)
We rewrite it to answer NXDOMAIN (CNAME .)

You could try this config example to see if it solves your issue:

rpz:
  name: “aaaaa.bbbbb.switch.ch."
  zonefile: “/var/lib/unbound/aaaaa.bbbbb.switch.ch.zone"
  master: 130.242.XXX.YYY@ZZZZ
  allow-notify: 130.242.XXX.YYY
  rpz-action-override: nxdomain <<—— this is the differentiator
  rpz-log: yes
  rpz-log-name: aaaaa.bbbbb
  tags: “malware”

HTH,
/P

7

Hi Fredrik,

Hi Hanspeter,

>
> > Hi Hanspeter,
> >
> > > Hi all,
> > >
> > > [replying to my own post]
> > >
> > > Apparently it is normal that unbound uses *a lot of RAM* after
> > > the
> > > initial load of the rpz zones (point 1 below).
> >
> > Does your RPZ zone contain a lot of records with the local data
> > RPZ
> > action? Due to the way the memory allocation is done here this
> > can
> > result in a very memory hungry Unbound instance. We are working
> > on a
> > fix
> > for this.
>
> I am not entirely sure what "local data RPZ action" means. almost
> all
> our records in the rpz zones are CNAMES.

(All RPZ actions use CNAME <data>)

"Local data” action means that the RPZ zone you’re supplying has an
“alternative” answer that’s presented to querying client, redirecting
the client to another host. (This explains why I don’t see your
unbound's memory-hogging behaviour on SUNET unbound instances.)
We rewrite it to answer NXDOMAIN (CNAME .)

ah, I understand, thanks for clarifying.

You could try this config example to see if it solves your issue:

rpz:
  name: “aaaaa.bbbbb.switch.ch."
  zonefile: “/var/lib/unbound/aaaaa.bbbbb.switch.ch.zone"
  master: 130.242.XXX.YYY@ZZZZ
  allow-notify: 130.242.XXX.YYY
  rpz-action-override: nxdomain <<—— this is the
differentiator
  rpz-log: yes
  rpz-log-name: aaaaa.bbbbb
  tags: “malware”

I tried overriding the action to nxdomain, as suggested. I didn't
change anything, unfortunately. still getting the memory leak.

Best,
Hp