Unbound freezes during RPZ reload

reduzent · September 25, 2026, 1:31pm

We run Unbound on Debian Trixie (13.5) on two servers as resolvers for the internal network. We also use Unbound to serve RPZs. Recently, Debian upgraded Unbound from 1.22 to 1.26 and since then we intermittently get freezes for 20-30 seconds, during which Unbound does not respond to any requests. I could trace back the freezes to certain RPZ updates. While Investigating the issue more, I found that Version 1.25.2 introduced a change in RPZ update behaviour. Up until 1.22, the internal policy structure for RPZs was built during start of the server. Subsequent IXFR were applied quickly without any noticable side-effects. With 1.26, it appears that Unbound forces an AXFR after four IXFR, which leads to a rebuild of the internal policy structure. During this rebuild Unbound does not answer any requests. One of the RPZs in question has ~2 million records and it takes ~24 seconds to rebuild during which any requests stay unanswered.

The change that introduced the new behavior is 1354624b. It seems that number of IXFR before an AXFR happens cannot be changed in the configuration and is hard-coded in:

#define NUM_IXFR_BEFORE_AXFR 5

We reverted to 1.22.

Is there a way we can upgrade to the most recent version without experiencing intermittent freezes?

Regards,
Roman Haefeli

reduzent · September 25, 2026, 1:34pm

There is an issue already open that seems related:

github.com/NLnetLabs/unbound

Unbound with RPZ enabled, stops answering client queries then loading new RPZ zones after AXFR

opened 08:32AM - 26 May 26 UTC

pettai

**Describe the bug** *Auth/RPZ zone load blocks query serving* _**Symptom**_ …When a slaved auth-zone or RPZ zone is (re)loaded via AXFR, unbound stops answering client queries for the duration of the apply. The stall scales linearly with zone size; multi-hundred-thousand-entry RPZ blocklists cause multi-second outages on every refresh. IXFR is less affected (delta-sized work); AXFR and HTTP-fetched zones are the pathological cases. _**Root cause**_ Three independent design facts in `services/authzone.c` combine to freeze the entire worker fleet, not just the worker doing the transfer. **To reproduce** Steps to reproduce the behavior: 1. in unbound, configure RPZ with a master server to fetch the RPZ zones from 2. wait for zone transfers for the RPZ zones to occur, specifically for full refetching with AXFR 3. from a DNS client, try asking unbound DNS queries over a non-UDP port (TCP/DoT/DoH/ProxyV2) and notice **Expected behavior** Unbound shouldn't throw away DNS queries on non-UDP ports then loading RPZ zones **System:** - Unbound version: 1.25.1 - OS: Ubuntu - `unbound -V` output: ``` Version 1.25.1 Configure line: --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/x86_64-linux-gnu --libexecdir=${prefix}/lib/x86_64-linux-gnu --disable-maintainer-mode --disable-dependency-tracking --disable-rpath --with-pidfile=/run/unbound.pid --with-rootkey-file=/var/lib/unbound/root.key --with-libevent --with-libhiredis --with-libnghttp2 --with-pythonmodule --enable-cachedb --enable-subnet --enable-dnstap --enable-systemd --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --libdir=/usr/lib Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.0.2 15 Mar 2022 Linked modules: dns64 python cachedb subnetcache respip validator iterator ``` **Additional information** _**Claude code analysis says:**_ 1. All auth/RPZ transfers run on worker 0 `daemon/worker.c` gates auth-zone task pickup behind `worker->thread_num == 0`: ``` if(worker->env.auth_zones #ifndef THREADS_DISABLED && worker->thread_num == 0 #endif ) auth_xfer_pickup_initial(worker->env.auth_zones, &worker->env); ``` Probe, transfer, and apply for every zone run on worker 0's event loop. While `apply_axfr` is iterating chunks and inserting RRs, worker 0 services no sockets — every client landing on worker 0 is blind. 2. `z->lock` write-lock is held for the entire destructive rebuild `xfr_process_chunk_list` acquires `z->lock` as a `wrlock` and holds it until the very end, around: - `traverse_postorder(&z->data, auth_data_del, NULL)` — frees the old tree - `rpz_clear(z->rpz)` — destroys and recreates all five RPZ sub-trees - per-RR `az_insert_rr_decompress` + `rpz_insert_rr` for every record - `auth_zone_verify_zonemd` — RRSIG/digest verification across the zone - `rpz_finish_config` — rebuilds parent pointers on three address-trees There is no atomic-swap pattern. The old tree is freed before the new one is built, so the `wrlock` must be held end-to-end. 3. RPZ policy lookup multiplies the wrlock into a fleet-wide stall Every worker consults RPZ policy on every query through three call sites in `services/rpz.c` (` rpz_apply_maybe_clientip_trigger, rpz_callback_from_iterator_module, rpz_callback_from_iterator_cname `). All three follow the same pattern: ``` lock_rw_rdlock(&az->rpz_lock); for(a = az->rpz_first; a; a = a->rpz_az_next) { lock_rw_rdlock(&a->lock); // blocks while worker 0 holds wrlock ... } ``` While worker 0 holds `z->lock` write on any one RPZ zone, every worker doing an iterator/CNAME callback blocks on `a->lock` read for that zone. With multiple configured RPZ zones the effect compounds — a single zone's apply freezes the RPZ subsystem across the whole worker fleet, not just worker 0.

reduzent · September 25, 2026, 1:39pm

I see how the mentioned change addresses an issue with growing memory usage. For us, however, memory usage is of much less concern than responsiveness of the resolver.

Yorgos · September 28, 2026, 8:01am

Hi Roman,

We are already working on improvements for rpz/auth-zone loading/XFR.
RPZ load performance improvements by wcawijngaards · Pull Request #1399 · NLnetLabs/unbound · GitHub,
Simdzone for auth zone file parsing by wcawijngaards · Pull Request #1397 · NLnetLabs/unbound · GitHub.
These will lower loading times.
There is also recently started work to offload to a separate thread.

There is a more involved workaround that you can use now if you are interested, via the fast_reload feature (i.e., unbound-control fast_reload).
For this you would have to:

Configure the rpz zone with only a zonefile, no primary servers,
Fetch the zone file outside of the running Unbound (e.g., dig @<server> AXFR <zone>), and
Issue the fast_reload command when the file is updated.

reduzent · September 28, 2026, 9:24am

There is also recently started work to offload to a separate thread.

That sounds sensible.

I’ll check your suggestion with the fast_reload feature in the meanwhile. Thanks a lot for your help.

reduzent · October 6, 2026, 6:38am

Hi Yorgos

Thanks to the new unbound-control fast_reload we were able to upgrade to 1.26.1 without fearing stalls due to RPZ reloading.

Here some observations I made. The size of all RPZs we use is 409 MB (in BIND text format). When loaded into memory with Unbound through the command unbound-control fast_reload, the size expands to 5.6 GB in memory. On the second unbound-control fast_reload memory consumption of Unbound doubles to 11.3 GB - which is probably expected since the internal policy structure needs to be set up a second time for a seamless transition.

Now comes the hairy part. When Unbound is restarted with systemctl restart unbound.service or the server is rebooted, it creates the internal policy structure directly at startup from the available zone files and Unbound memory consumption is at 5.6 GB after the restart. A subsequent unbound-control fast_reload increases memory usage to 11.3 GB. Now, another run of unbound-control fast_reload again increases the memory to 16.9 GB.

In other words: When I let Unbound the read the complete RPZ files at startup, the memory consumption is three times the internal policy structure. In order to save the third copy, I need to make sure that RPZ files are empty at Unbound start and that I only load RPZ policies with unbound-control fast_reload.

Why does Unbound not release memory for internal policy structures that it apparently does not use anymore?

Kind regards,

Roman