/etc/hosts handling plugin for unbound

Unbound
1

Hi everyone,

Is there some plugin for automatically watching /etc/hosts file for changes and loading them as a local data?

I am thinking about supporting unbound as a default localhost cache. But I think many people rely on /etc/hosts changes are propagated automatically to the cache. Dnsmasq and systemd-resolved both try to watch this file and re-read it after changes.

Is there any existing way to populate local data automatically from it? Is it already possible to synchronize local data in unbound with legacy text file?

Regards,
Petr

2

Any real software uses /etc/nsswitch.conf which states something like:

hosts: files myhostname mdns4_minimal [NOTFOUND=return] dns

The "files" directory (see man nsswitch.conf) reads in /etc/hosts first.

This happens before the "dns" entry, so before unbound is used. So for
apps on localhost this should work fine? It is always read (and not
cached)

But I guess you want to make it systemd-resolved+glibc hack compatible,
where calls to gethostbyname() are intercepted outside of nsswitch by a
linux glibc hack, then fed to systemd-resolved, which I guess might do the
wrong thing by using dns before files ?

Or are you trying to work with the libunbound call of:

         /* lookup from /etc/hosts before DNS lookups as people expect that */
         ugh = ub_ctx_hosts(dns_ctx, "/etc/hosts");

and are afraid changes in /etc/hosts are not picked up without an
unbound restart?

I guess I feel /etc/hosts is there only for localhost apps in case of
broken DNS. The days you could leave something out of DNS by putting in
the /etc/hosts file are kinda long gone.

Perphaps you can detail a bit more of what you are looking for and what
you are afraid of ?

Paul

3

Is there some plugin for automatically watching /etc/hosts file for changes and loading them as a local data?

I am thinking about supporting unbound as a default localhost cache. But I think many people rely on /etc/hosts changes are propagated automatically to the cache. Dnsmasq and systemd-resolved both try to watch this file and re-read it after changes.

Is there any existing way to populate local data automatically from it? Is it already possible to synchronize local data in unbound with legacy text file?

Any real software uses /etc/nsswitch.conf which states something like:

hosts: files myhostname mdns4_minimal [NOTFOUND=return] dns

The "files" directory (see man nsswitch.conf) reads in /etc/hosts first.

This happens before the "dns" entry, so before unbound is used. So for
apps on localhost this should work fine? It is always read (and not
cached)

That is not strictly true. Some software may use DNS explicitly via specialized libraries, like libresolv, ldns, libunbound, etc. dig and host tools are the best examples. What you wrote applies only to applications using getaddrinfo() or gethostbyname() API calls.

But I guess you want to make it systemd-resolved+glibc hack compatible,
where calls to gethostbyname() are intercepted outside of nsswitch by a
linux glibc hack, then fed to systemd-resolved, which I guess might do the
wrong thing by using dns before files ?

I am more inspired by dnsmasq, which I maintain also. But both of those is able to watch /etc/hosts and auto-load its entries into the local DNS cache. I think it might be useful in some cases to have very simple way to add address override for some names on whole machine. I use it sometimes to create common records for virtual machines or containers running on my machine. I think it is convenient to query those names also from DNS only tools like dig, unbound-host or similar. dnsmasq spawned for libvirt forwards queries to my localhost cache, because /etc/resolv.conf on the host points to it.

Sure, it should be possible to disable this behavior. An unbound module or plugin might be a way.

Or are you trying to work with the libunbound call of:

    /\* lookup from /etc/hosts before DNS lookups as people expect that \*/
    ugh = ub\_ctx\_hosts\(dns\_ctx, "/etc/hosts"\);

and are afraid changes in /etc/hosts are not picked up without an
unbound restart?

Yes something similar. But this requires explicit code in (every) client binary. Instead of a common code in the localhost cache instance.

I guess I feel /etc/hosts is there only for localhost apps in case of
broken DNS. The days you could leave something out of DNS by putting in
the /etc/hosts file are kinda long gone.

Is that true? Can you give an example, why it is so? Isn't it the argument for adding /etc/hosts to the DNS then?

Perphaps you can detail a bit more of what you are looking for and what
you are afraid of ?

Paul

I just would like ability to provide a way both dnsmasq and systemd-resolved have. Yes, I know it is possible to use unbound-control to add local zone and local data into it. But that is too complicated for ordinary user IMO. It is not persistent. sudoedit /etc/hosts is simple enough even for (a bit) advanced user. Should be relatively simple to implement also. It should not break anything if enabled by default on workstations.

4

Unbound has some function to load local data dynamically such as “local-data” configuration or RPZ, or modifying DNS response with Python script. So if somebody (including me) wants to load hosts file to Unbound, they will just write a script that fulfills their requirements.

I understand that ability to loading /etc/hosts as local data (by just giving /etc/hosts path) is useful for some people, but I don’t know how the function should be and fulfills many users’ requirements. Needs to add PTR records? Needs to add some domain suffix?

Petr Menšík via Unbound-users <unbound-users@lists.nlnetlabs.nl>:

5

Hi,
attached you will find a script I use to update my /etc/hosts ads block list
and add it to unbound as local-data on debian/devuan. Hope it could help
as inspiration (yes I know it is rather ugly but it worked reliably for years
for me as crontab script).

Ciao,
Tito

(attachments)

adaway.sh (5.95 KB)

6

This happens before the "dns" entry, so before unbound is used. So for
apps on localhost this should work fine? It is always read (and not
cached)

That is not strictly true. Some software may use DNS explicitly via specialized libraries, like libresolv, ldns, libunbound, etc.

Like libunbound, they can offer that functionality.

dig and host tools are the best examples.

The host command was obsoleted before the ifconfig command was obsoleted.

The dig command is a dns specific tool.

I am more inspired by dnsmasq, which I maintain also. But both of those is able to watch /etc/hosts and auto-load its entries into the local DNS cache.

You mean not just to “cache” for local host but to serve for the network ? For that I would use inbound-control to feed it.

I think it might be useful in some cases to have very simple way to add address override for some names on whole machine. I use it sometimes to create common records for virtual machines or containers running on my machine.

You can drop these in /etc/unbound.d/ ?

Sure, it should be possible to disable this behavior. An unbound module or plugin might be a way.

Instead of using your command to add the entry to /etc/hosts, wrap it in an unbound command to either inject in the running daemon or put it in the .d directory if you want it to persist. You could reload/restart unbound on each change if you are just serving the local machine ?

I guess I feel /etc/hosts is there only for localhost apps in case of
broken DNS. The days you could leave something out of DNS by putting in
the /etc/hosts file are kinda long gone.

Is that true? Can you give an example, why it is so? Isn't it the argument for adding /etc/hosts to the DNS then?

I think it’s a reason to stop using /etc/hosts
For fedora, I insured you could configure local data in a persistent way using the .d directory.

People putting thousands of entries in /etc/hosts for anti-spam and ad blocking is also not the best and fully supported using the .d directly with unbound.

I just would like ability to provide a way both dnsmasq and systemd-resolved have. Yes, I know it is possible to use unbound-control to add local zone and local data into it. But that is too complicated for ordinary user IMO.

Then I guess write a systemd ExecStartPost= option for unbound to loop over /etc/hosts and run unbound-control for the user ? I would approve such a change for the fedora package.

It is not persistent. sudoedit /etc/hosts is simple enough even for (a bit) advanced user. Should be relatively simple to implement also. It should not break anything if enabled by default on workstations.

I think this would be covered by the above change.

Paul

7

Unbound has some function to load local data dynamically such as "local-data" configuration or RPZ, or modifying DNS response with Python script. So if somebody (including me) wants to load hosts file to Unbound, they will just write a script that _fulfills their requirements_.

I understand that ability to loading /etc/hosts as local data (by just giving /etc/hosts path) is useful for some people, but I don't know how the function should be and fulfills many users' requirements. Needs to add PTR records? Needs to add some domain suffix?

Yes, reverse PTR record back to primary name would be nice. Domain suffix should not be necessary. /etc/hosts allows aliases, so you can have line like "10.0.0.1 primary.example.com primary". That would make primary.example.com a primary fully qualified domain name. And "primary." just an alias without matching PTR record. Either just address or even CNAME to primary.example.com.

It seems to me it could be a special implementation of Cache DB module. I admit I have never tried to use CacheDB module yet.

8

Domain suffix should not be necessary. /etc/hosts allows
aliases, so you can have line like "10.0.0.1 primary.example.com primary". That would make primary.example.com a primary
fully qualified domain name. And "primary." just an alias without matching PTR record.

Would it be a TLD "primary.", or would it be primary.<yourdomainsuffix>. ?

It's tricky loading /etc/hosts into a resolver for unqualified entries.
I kinda hope that unbound would just ignore them. A quick test shows
it will just override a real FQDN. So on my machine with "search nohats.ca"
in /etc/resolv.conf, an entry in /etc/hosts for "www" will use whatever
is in /etc/hosts and not what is in dns for www.nohats.ca.

It seems to me it could be a special implementation of Cache DB module. I admit I have never tried to use CacheDB module
yet.

Not sure if this is worth the energy, when a simple systemctl restart unbound
would also reread /etc/hosts. Sure you lose your cache, but I lost that
battle a long time before when every interface change results in cache
wipe.

Paul

9

Would it be a TLD "primary.", or would it be primary.<yourdomainsuffix>. ?

It's tricky loading /etc/hosts into a resolver for unqualified entries.
I kinda hope that unbound would just ignore them. A quick test shows
it will just override a real FQDN. So on my machine with "search nohats.ca"
in /etc/resolv.conf, an entry in /etc/hosts for "www" will use whatever
is in /etc/hosts and not what is in dns for www.nohats.ca.

I think /etc/resolv.conf is configuration exclusively for dns plugin in nsswitch/glibc. Any search inside does not apply to hosts entries. If you specified www in /etc/hosts, then why shouldn't it reply to such name? Whatever were specified should be also loaded. It might be configurable behavior for single label names, but I would serve also such names by default.

It seems to me it could be a special implementation of Cache DB module. I admit I have never tried to use CacheDB module
yet.

Not sure if this is worth the energy, when a simple systemctl restart unbound
would also reread /etc/hosts. Sure you lose your cache, but I lost that
battle a long time before when every interface change results in cache
wipe.

Paul

I have one picture here in office related to it. I think if all networks specify domains with local specific content and wipe only those (sub)domain cache entries on disconnection (or re-connection), then the rest of the cache do not have to be erased. But until we have working protocol to advertise such subdomains automatically and trustworthy, automatic cache flush is safe enough. Network Manager does not yet support any RFC related to ADD IETF proposals related to it. Any other system does not seem to support something similar either. On common end-user device flushing whole cache is not a big issue IMO. Should be avoided for servers however.