What is considered the best reply from a local cache to avoid traffic
to the root servers for domains that are reserved (for local use etc.,
such as .home, .corp) or those you wish to block?
Is always_nxdomain better than always_refuse? Is there an even better reply?
Does one or the other prevent the querying system from attempting to
query other caches, that may or may not be blocked?
RFC 6761 explains how special-use domain names are registered, and part of
each registration is a description of how various kinds of DNS software
should handle the special name. So in many cases you can consult the IANA
registry for pointers to RFCs that say how your server should work.
I used to have a fairly elaborate configuration that returned NXDOMAIN for
lots of special-use domains, but I dropped all that rubbish when RFC 8198
NSEC negative answer synthesis became a thing. A locally-served root zone
gets you some of the same benefits.
NXDOMAIN is generally safer than REFUSED because there's less risk of
provoking clients to make useless retries.
There's one complication that I know of: RFC 6762 says that recursive
servers should reply with NXDOMAIN for .local, but Avahi (a Linux
implementation of mDNS) tries to work out if .local is a real zone and if
so it stops doing mDNS, which is not nice when it is an unexpected
side-effect! If unbound's always_nxdomain returns NXDOMAIN for everything
_including_ the local-zone's apex (i.e. .local itself must be NXDOMAIN)
then you will be OK. (I can't tell from the docs if this is the case.)
If that's the actual end goal, then answering a different question in
the hope that it's more useful to you:
RFC 8806 Running a Root Server Local to a Resolver
Appendix B.4. Example Configuration: Unbound 1.9
Secondary the root zone on the resolvers and the traffic to the roots
scales with what's legitimate and with update frequency, instead of with
all the garbage various systems spit out.
At which point, Unbound replies NXDOMAIN for _any_ unrecognized TLD not
otherwise configured in it.