Seeking advice for deploying an anycast cluster

NSD
1

Hello,

I'm trying to use an anycast constellation to affect a "hidden
primary" type of configuration with NSD.

I'm trying to figure out the best way for constellation members to
cluster data. Seems like AXFR might be useful but I'm trying to
figure out how to set NSD up on a timer to automatically AXFR from the
primaries every X minutes. Is there a way to do that?

Of note, using NOTIFY doesn't seem possible, because if I start
sending NOTIFYs to a constellation member, there doesn't seem to be a
way to propagate that other constellation members.

Best,
Daniel

2

Hello,

Of note, using NOTIFY doesn't seem possible, because if I start
sending NOTIFYs to a constellation member, there doesn't seem to be a
way to propagate that other constellation members.

This is usually solved by sending NOTIFY messages not to the anycasted
cluster address but to all the unique address of each cluster node instead.

3

The main issue I'm running into is I want to keep the primary's
interface to the world as simple as possible. At maximum, two hosts
to communicate with. So for that to work, I'd need to somehow
cluster my NSD instances together or I'd need some sort of proxy
server that can listen for incoming NOTIFYs and then distribute them
to the rest of the constellation. I don't think any of the usual
suspects (nginx, haproxy, etc) have that capability out of the box.
Do they?

4

First, the advice from Ondřej Caletka is worth heeding.

The main issue I'm running into is I want to keep the primary's
interface to the world as simple as possible.

Fair enough.

At maximum, two hosts to communicate with.

If this is a limit you have freely chosen, I suspect you may be
importing inapplicable experience from another problem domain,
and would advise reviewing your choice.

NSD is well capable of, and easily configured for, handling much
greater fanout than two. In case the anycast cloud is really
enormous, it may be sensible to use some intermediate distribution
servers, as described below using the term "hidden slave".

So for that to work, I'd need to somehow
cluster my NSD instances together or I'd need some sort of proxy
server that can listen for incoming NOTIFYs and then distribute them
to the rest of the constellation.

Any of the authoritative name-server codes is designed to do exactly
this kind of proxying. The old-fashioned terminology for a server
which does only this, and which is not announced to the world in an
NS record, is "hidden slave". Setting one of these up is particularly
easy to do with (perhaps another instance of) NSD. Other choices might
be BIND (named), Knot, or PowerDNS.

An NSD-based hidden slave needs to refer to the upstream master
in 'allow-notify' and 'request-xfr' configuration directives and to
refer to each dependent downstream server in 'notify' and 'provide-xfr'
directives. Of course, the downstream servers must be configured
correspondingly to accept NOTIFY and request a zone transfer when
appropriate.

I don't think any of the usual
suspects (nginx, haproxy, etc) have that capability out of the box.
Do they?

I don't know: modules of all kinds abound; but it would be really
extraordinary to use such a web proxy as part of a DNS infrastructure.

  [...]

This is usually solved by sending NOTIFY messages not to the anycasted
cluster address but to all the unique address of each cluster node instead.

I may have mentioned already that this advice from Ondřej Caletka
is worth heeding.

Niall O'Reilly

5

Thank you for taking the effort to type this up. I think you've hit
on a solution here. At least I believe you've got me thinking in the
right direction now.

NSD is well capable of, and easily configured for, handling much
greater fanout than two. In case the anycast cloud is really
enormous, it may be sensible to use some intermediate distribution
servers, as described below using the term "hidden slave".

It is. 38 servers and counting. Spread out over the globe.

> So for that to work, I'd need to somehow
> cluster my NSD instances together or I'd need some sort of proxy
> server that can listen for incoming NOTIFYs and then distribute them
> to the rest of the constellation.

Any of the authoritative name-server codes is designed to do exactly
this kind of proxying. The old-fashioned terminology for a server
which does only this, and which is not announced to the world in an
NS record, is "hidden slave". Setting one of these up is particularly
easy to do with (perhaps another instance of) NSD. Other choices might
be BIND (named), Knot, or PowerDNS.

I've been thinking about using PowerDNS anyways to front a web
interface for my customers. Seems like the shortest path to market
considering it supports pgsql out of the box.

An NSD-based hidden slave needs to refer to the upstream master
in 'allow-notify' and 'request-xfr' configuration directives and to
refer to each dependent downstream server in 'notify' and 'provide-xfr'
directives. Of course, the downstream servers must be configured
correspondingly to accept NOTIFY and request a zone transfer when
appropriate.

So I'm worried about nsd getting the right information from the right
source. If I put my nsd instances in a "full mesh" so to speak,
where they are all XFR and NOTIFY each other, my "A" and "B" PowerDNS
servers would still need to be the authoritative source for
information. I'm assuming that's where the serial number of the zone
comes into play? Highest provided serial number wins, right?

6

So I'm worried about nsd getting the right information from the right
source. If I put my nsd instances in a "full mesh" so to speak,
where they are all XFR and NOTIFY each other, my "A" and "B" PowerDNS
servers would still need to be the authoritative source for
information. I'm assuming that's where the serial number of the zone
comes into play? Highest provided serial number wins, right?

Correct.

Don't forget that you would be wise to include TSIG as well. Here, between
servers managed by the same entity, it is child's play. And being able
to prove that the AXFR is correct all the way is quite beneficial.

Further, you need a system to manage the "new zone deployment" process,
as well as removal (but adding is more time-critical than removal). I've
used "dper.pl" from Kirei (https://github.com/kirei/dper) to automate
this for quite a few years now. No hickups. A new version that is Python
and YAML is in the works, of which I will have experience soon.

In summary:

* every slave should fetch its zone from at least 2 masters,

* serial governs which zone is loaded and served,

* TSIG set up between masters and slaves,

* NOTIFY (by virtue of "also-notify" in BIND and PowerDNS syntax) makes
  things fast,

* Zone deployment is and was always supposed to be OOB, and needs to
  be handled. (NB: There with some frequency pops up suggestions to do
  in-band provisioning, and I THINK I've seen it on the feature list for
  some implementations, but no standard method has yet been devised.)

rgds,

7

This sounds like you want to have multiple NSDs at a single location, having load balancing between them. How do you deploy your servers? For anycast you need BGP anyways to announce your anycast prefixes. Hence just announce the prefixes from all your NSD nodes if the provider supports ECMP (equal-cost multipath) BGP.

Or if you deploy also routers, you can easily use OSPF.

This is what we do.

regards
Klaus

8

Hello Måns!