.PR servfails with Unbound but not with BIND

Unbound
1

% dig SOA pr.

; <<>> DiG 9.5.1-P3 <<>> SOA pr.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 940
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pr. IN SOA

;; Query time: 21 msec
;; SERVER: ::1#53(::1)
;; WHEN: Tue Sep 8 17:06:58 2009
;; MSG SIZE rcvd: 31

% dig +cd +dnssec SOA pr.

; <<>> DiG 9.5.1-P3 <<>> +cd +dnssec SOA pr.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62970
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 2, AUTHORITY: 6, ADDITIONAL: 7

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;pr. IN SOA

;; ANSWER SECTION:
pr. 86150 IN SOA pascal.nic.pr. tech.nic.pr. 2009090836 1200 900 1209600 3600
pr. 86150 IN RRSIG SOA 5 1 86400 20091008120105 20090908120105 62607 pr. VIiEZemSfxbg23K3vROW8ZEB2QYoVfcfqP+gZ0Ge88MsIDVfx5NZpP1t b0PDUa9hTRClxeWdK6ofzYB09YVLlSNC9FsM1Qrr17M+taTMid02tEyv sSQmUayY3GuN2UftFnsKpDVzVP39m/2uQP32hgDQLYJns3Kw9/8wiQH1 Cbk=

;; AUTHORITY SECTION:
pr. 86076 IN NS pr-dns.denic.de.
pr. 86076 IN NS descartes.nic.pr.
pr. 86076 IN NS pr-ns.anycast.pch.net.
pr. 86076 IN NS golomb.nic.pr.
pr. 86076 IN NS pascal.nic.pr.
pr. 86076 IN RRSIG NS 5 1 86400 20091008120105 20090908120105 62607 pr. gSDrvv6tK5tKyMq5hvSdbwsmtiopjGlSKxoOP/AqS6YzfWVl6WF96eX/ oa5EW1Uu0cLpJLhvBp4jYFDGMyBXGL6uIKn4HoC7d2eUyREhJR/a7KRo GcJ5oyyc6rr/449ou1tZYsIdl+wNZ0Kv6mZ27C1gUC+VqzsF96B7QQr8 PrI=

;; ADDITIONAL SECTION:
golomb.nic.pr. 281 IN A 134.202.6.100
pascal.nic.pr. 281 IN A 134.202.0.120
descartes.nic.pr. 281 IN A 134.202.1.120
golomb.nic.pr. 281 IN RRSIG A 5 3 300 20091008120136 20090908120136 25188 nic.pr. WSg4gWYk2E7CNlz5OO+r9ucUuyzxKfO4iLls1U/bVllASRNW+AUPrOBI A6w9aAAoCNyvvDcUISauGTFfPZItsd/1SEB+YrIoddVq8k+HhKEJa0oO 5a/E5DUagRl8u1AhS4SLFXoH+6dOqMV20fKEnFvzTJ6hkJAQFaduthwj U2c=
pascal.nic.pr. 281 IN RRSIG A 5 3 300 20091008120136 20090908120136 25188 nic.pr. abpgt+cI54VYatNI16UTOnouVowHUqiwb5SAbq6j0n4xVxmrMLTTaUG5 QzdbluTMhykCfbkX2RQzq5no5aM8U8cXQPJ3MNBAtPzzTzsOkrV7hanc M3YVT10dm5odwNk8cK1ks+S0yFwv8WlRcdJUeqqOUjeTkr1SZ0NXjG9q VV4=
descartes.nic.pr. 281 IN RRSIG A 5 3 300 20091008120136 20090908120136 25188 nic.pr. mIaBbAIVL5F7K3VSfqcZq2A43aDsOZCI9BfyywTAvf/gXYWbAiCRKSDL gtEvSSU0cG54N8IrYNk1y8Epu+PZp4aIPpbhsUlGDzwGD2MsqlvelIOG j9EXfYrpxVYQLynIkmNT4STwf2YRRWoAJ5tqk+U3qWJ2JRH59OVZq2l3 QJ0=

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Tue Sep 8 17:07:17 2009
;; MSG SIZE rcvd: 1076

I get the key through DLV.

% dig DLV pr.dlv.isc.org.

; <<>> DiG 9.5.1-P3 <<>> DLV pr.dlv.isc.org.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48135
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 6, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pr.dlv.isc.org. IN DLV

;; ANSWER SECTION:
pr.dlv.isc.org. 3255 IN DLV 62704 5 2 57E017A982196D194B3F52CDD39F86A9A33DED75064F285A9242BA7A 448A659C
pr.dlv.isc.org. 3255 IN DLV 62704 5 1 AFA72CB11D4C97657D82338AF6D569ED614166EB

;; AUTHORITY SECTION:
dlv.isc.org. 3570 IN NS dlv.sfba.sns-pb.isc.org.
dlv.isc.org. 3570 IN NS dlv.ams.sns-pb.isc.org.
dlv.isc.org. 3570 IN NS ns1.isc.ultradns.net.
dlv.isc.org. 3570 IN NS ns.isc.afilias-nst.info.
dlv.isc.org. 3570 IN NS dlv.ord.sns-pb.isc.org.
dlv.isc.org. 3570 IN NS ns2.isc.ultradns.net.

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Tue Sep 8 17:07:38 2009
;; MSG SIZE rcvd: 290

% dig +cd DNSKEY pr.

; <<>> DiG 9.5.1-P3 <<>> +cd DNSKEY pr.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26009
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pr. IN DNSKEY

;; ANSWER SECTION:
pr. 86045 IN DNSKEY 256 3 5 BQEAAAABvS8Q64q8v62DW3y4EtUmsHr0dpU9Mizo63NXFMlEA4UaO88s B5il79MbJ0dzmRZ7M+j/E5pVSTTazJsK6LMnncBF3bwMWo4/nVVB0d9E 6CsClsJFU+A0a8kWIZ+aXuqUHO7QZ88qG7cwLbTNwHeo1X+ArvXgXmU6 OaemL3v5+eU=
pr. 86045 IN DNSKEY 257 3 5 AwEAAeDPv9lQ7Ej5Ld9Fz/FKLhdOajwtEXsWykj65ugIa4Di1nY6ti9n dkeR4kp1aSNlvf6N7KsjunfMJj4SccBwcY77DrxmQ+g9nI09ePMZvxF2 U63Lv9BftGaIguYdkYZVSwHd1q7DdXqNkLaD4tZEHiN0h/3wBdTQUPH1 IoskD1vGxiPw2egftk6sVQdvOJWaAgSpmG0eq+/e90WVTNX4/xhA17Pr dQQJIheZQ3+EsDoil8kyJZC12KoHYpFklx7+aCiR2u8Fumy6ARFR4PP0 n7bnBaKOgMpVzz+KI79a3USDkj9RhNog50iSWgaBM75Xu0IBNEpcCVYZ YjwDESgiDXc=
pr. 86045 IN DNSKEY 256 3 5 AwEAAZpi4OSTiCcMfLUCj+d8Ps3xB1egdn38I2EdHxoKxutS79EAHdFT iauogWygbKLhOUYgorqWzM8XBMkXabN7tof6SDiL7v13NL7rr1TuG1GY GOh66ZBF8BpGieufNb3Fzvc3addcbkw0iAf6bsbBgOWJ74xOErPPYGUs TPRcUfor

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Tue Sep 8 17:07:48 2009
;; MSG SIZE rcvd: 605

Unbound 1.3.2. A BIND resolver has no problem.

2

[added dnsop@ietf.org to the reply]

Subject: [Unbound-users] .PR servfails with Unbound but not with BIND

% dig SOA pr.

I get the key through DLV.

It's outdated and wrong and missing the new key.

On Aug 19 2009, pr added this key:

PR. IN DNSKEY 257 3 5 AwEAAeDPv9lQ7Ej5Ld9Fz/FKLhdOajwtEXsWykj65ugIa4Di1nY6ti9n

dkeR4kp1aSNlvf6N7KsjunfMJj4SccBwcY77DrxmQ+g9nI09ePMZvxF2
U63Lv9BftGaIguYdkYZVSwHd1q7DdXqNkLaD4tZEHiN0h/3wBdTQUPH1
IoskD1vGxiPw2egftk6sVQdvOJWaAgSpmG0eq+/e90WVTNX4/xhA17Pr
dQQJIheZQ3+EsDoil8kyJZC12KoHYpFklx7+aCiR2u8Fumy6ARFR4PP0
n7bnBaKOgMpVzz+KI79a3USDkj9RhNog50iSWgaBM75Xu0IBNEpcCVYZ
YjwDESgiDXc=

And on Sep 4 2009, pr removed this keys:

< PR. IN DNSKEY 257 3 5 AwEAAc6SkFSHw00wJFUWd1Td/efsxhfX+UTrxrzqQXNuZ8Qj2PiP6p/m
BxysJt06XgSCB41CPhkgvgqrtdaJ/hXKG81xNXUcGfqvV9wYMJnN+oBB
/lLaQU/39fWaNc4fBGiRI2dNDVKPry2YX6y04YrEGRM+wf6HWHVdW1Js
xuMuDOSr

% dig DLV pr.dlv.isc.org.

;; ANSWER SECTION:
pr.dlv.isc.org. 3255 IN DLV 62704 5 2 57E017A982196D194B3F52CDD39F86A9A33DED75064F285A9242BA7A 448A659C
pr.dlv.isc.org. 3255 IN DLV 62704 5 1 AFA72CB11D4C97657D82338AF6D569ED614166EB

These are the old key, and that DLV record should be removed. The new DLV record should be:

pr.dlv.isc.org. IN DLV 6277 5 2 6966580bb25c608540e8224039561c7b2a1488d1f927c5cdbd137f4ef3d31528
pr.dlv.isc.org. IN DLV 6277 5 1 05d02dce8385974d958a5db409f6ff3658293b2

I guess we need a MUCH better communication method between TLD's, iTAR and ISC's DLV. This is bad.

Paul

3

a message of 33 lines which said:

Out of curiosity (since I'm not on the unbound-users list), why did it
work with BIND and not Unbound?

Probably a caching effect and not a real difference between the
resolvers.

My BIND resolver now fails as well.

4

a message of 126 lines which said:

% dig SOA pr.

; <<>> DiG 9.5.1-P3 <<>> SOA pr.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 940

It works now. The DLV registry at ISC updated the key. Apparently, the
.PR people rolled over with a very short notice and anyone using DLV
or manual tracking of keys will have experienced the problem.

Lesson learned: activating DNSSEC validation today is only for
playing and should not be done in a production environment.

5

.SE has been in production mode for the last 2.5 years. It has been working very well in Sweden with all the major resolver operators performing DNSSEC validaion. I would rather say that DLV is not ready for use in a production environment.

6

Stephane, Patrik,

a message of 126 lines which said:

% dig SOA pr.

; <<>> DiG 9.5.1-P3 <<>> SOA pr.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 940

It works now. The DLV registry at ISC updated the key. Apparently, the
.PR people rolled over with a very short notice and anyone using DLV
or manual tracking of keys will have experienced the problem.

Lesson learned: activating DNSSEC validation today is only for
playing and should not be done in a production environment.

.SE has been in production mode for the last 2.5 years. It has been working
very well in Sweden with all the major resolver operators performing DNSSEC
validaion. I would rather say that DLV is not ready for use in a production
environment.

I would rather say that .PR is at fault here. I have discovered that
.PR key has changed only from my automated ITAR update script. At
first they had removed .PR key from ITAR and after that they had added
new key - it didn't look like regular well planned rollover. Or had
anybody seen some announcement?

When .SE had changed their key, I got announcement from several places
and it was well planned and everybody could prepare before rollover
was done.

Also if .PR knew that their key is in DLV registry, that should
exchange their key in DLV as well.

Ondrej

7

Lesson learned: activating DNSSEC validation today is only for
playing and should not be done in a production environment.

.SE has been in production mode for the last 2.5 years.

I agree that the conclusion that DNSSEC is only for playing is very
outdated.

It has been working very well in Sweden with all the major resolver operators performing DNSSEC validaion. I would rather say that DLV is not ready for use in a production environment.

What are the Swedish ISP's loading into their resolvers? Just the .se
key? Or all the keys from iTAR?

Paul

8

As far as I know they only use the key from .SE. And I know that none of them use DLV.

9

Ondřej Surý wrote:

Also if .PR knew that their key is in DLV registry, that should
exchange their key in DLV as well.

I'd rather they just update it in the ITAR, but do so in a manner that
is not going to cause an outage.

If a TLD asks if they should put their keys both in ISC's DLV and the
ITAR, I try to get them to only put it in the ITAR, and we will get it
from there. .TH chose to manually manage the key in both places, I
don't know exactly why but it's no problem for us either way; we just
exclude .TH from our ITAR import if it exists in DLV already.

- --Michael

10

Ondřej Surý wrote:

Also if .PR knew that their key is in DLV registry, that should
exchange their key in DLV as well.

I'd rather they just update it in the ITAR, but do so in a manner that
is not going to cause an outage.

Ok, what I really wanted to say, that they should ensure that key in
DLV is correct.

BTW and looks like I misremembered emails from my auto ITAR script.
There was 16 day window of key overlap.

Ondrej

11

At first they had removed .PR key from ITAR and after that they had added
new key - it didn't look like regular well planned rollover.

No, they first added the new key to their zone on Aug 19. Then they removed
the old key from their zone on Sep 4.

But even now, they are still refering to the old key themselves at some
places, such as: http://dnssec.nic.pr/serverconf.php

(Also their SSL negotiation on https://dnssec.nic.pr/ is still failing)

Also if .PR knew that their key is in DLV registry, that should
exchange their key in DLV as well.

Yes. Anyone with a key in the DLV should make sure its updated before
they remove the old key. I too believe that's a mistake from the PR people.

Paul

12

Ondřej Surý wrote:

Also if .PR knew that their key is in DLV registry, that should
exchange their key in DLV as well.

I'd rather they just update it in the ITAR, but do so in a manner that
is not going to cause an outage.

If a TLD asks if they should put their keys both in ISC's DLV and the
ITAR, I try to get them to only put it in the ITAR, and we will get it
from there. .TH chose to manually manage the key in both places, I
don't know exactly why

Probably to avoid disasters like these.

but it's no problem for us either way; we just
exclude .TH from our ITAR import if it exists in DLV already.

- --Michael

Could we move this discussion away from the unbound-users list? We seem to digress. Is there a dlv-users list?

Roy

13

Not to follow up my own post, but I'd like to point out that .pr is not
the only problem in TAR-space right now.

- From the ARIN TAR import:

171.in-addr.arpa:
Fetching DNSKEYS from DNS for 171.in-addr.arpa
Unused DS for 171.in-addr.arpa, type RSASHA1/SHA-1, tag 54333

153.in-addr.arpa:
Fetching DNSKEYS from DNS for 153.in-addr.arpa
Unused DS for 153.in-addr.arpa, type RSASHA1/SHA-1, tag 35994

154.in-addr.arpa:
Fetching DNSKEYS from DNS for 154.in-addr.arpa
Unused DS for 154.in-addr.arpa, type RSASHA1/SHA-1, tag 49773

What this script does is compare data from three sources: what is
currently in ISC's DLV, what is in the TAR, and what is in the zone.
ISC's DLV will attempt to match the TAR's data: if a key is removed
from the TAR, we will remove it from DLV regardless if it is still in
the zone. We will attempt to add any new keys we find DS records for in
the TAR, if they exist in the zone.

In this case, I believe these three domains were delegated away from
ARIN, but they (and DS records) are still present in the ARIN TAR.

In this case, anyone who has a tar-import script would reject any data
from those domains, since the trusted-key would be configured, yet it is
not correct.

- --Michael

14

a message of 86 lines which said:

.SE has been in production mode for the last 2.5 years. It has been
working very well in Sweden with all the major resolver operators
performing DNSSEC validaion. I would rather say that DLV is not
ready for use in a production environment.

DNSSEC validation with just one manually configured trust anchor is
not too difficult. As long as they do not validate several
widely-different TLD, it is difficult to call that a real deployment,
with all its inherent difficulties.

15

a message of 41 lines which said:

I would rather say that .PR is at fault here.

I remember that they had expired signatures in the zone at a time. But
my point is not here: sure, some TLD perform better than others. But,
with regular DNS, you have to work really hard to completely break the
resolution process. With DNSSEC, it is the opposite, you have to be
very good to make things work.

Activating DNSSEC validation mean switching from "it will work most of
the time" to "it will work only when everything is OK". I'm not sure
that many people are ready to go this way.

16

a message of 6 lines which said:

Could we move this discussion away from the unbound-users list? We
seem to digress. Is there a dlv-users list?

I agree it is not (unlike what I said at the beginning) an Unbound
problem. But it is not a DLV issue either: the same problem could have
occured with any key delegation system (ITAR, a signed root, etc).

17

a message of 73 lines which said:

As far as I know they only use the key from .SE.

Since I work for a ccTLD, I approve this idea: security should only be
for the national TLD :slight_smile: