Unbound DNS over HTTPS Trouble

Hi,

I've been trying out DoH using Unbound 1.13.1 on a FreeBSD host and a Let's Encrypt TLS certificate. Unbound starts and listens on my DoH port, and when I connect to it, the TLS session is established as expected. I can send DNS queries and the server sends me a response, but it's one byte short and is simply a reply containing NO RR records, only the original question sent to the server, oddly truncated by a single byte.

For example, here's what happens when I query Cloudflare's 1.1.1.1 DoH server (using it like a control to compare Unbound's response) and then my local test Unbound server using the same query for an A record for google.com:

RAW QUERY, 28 BYTES:
58, 102, # Query ID
1, # qr=0 (request), opcode=0, aa=0,
# tc=0, rd=1 (recursion desired)
0, # ra=0, z=0, rcode=0
0, 1, # Number of questions: 1
0, 0, # Number of answers: 0
0, 0, # Authority RRs: 0
0, 0, # Additional RRs: 0
--- QUESTION 1 of 1 ---
6, 103, 111, 111, 103, 108, 101, # label "google"
3, 99, 111, 109, # Label "com"
0, # End of labels
0, 1, # Class "IN" (1)
0, 1 # Resource type "A" (1)

Cloudflare 1.1.1.1 using HTTP/2:
https://1dot1dot1dot1.cloudflare-dns.com/dns-query?dns=OmYBAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ==

HTTP/2 Response Headers:
Content-Type: application/dns-message
Content-Length: 44

RAW REPLY, 44 BYTES:
58, 102, # Query ID (matches question ID)
129, # qr=1 (answer), opcode=0, aa=0,
# rc=0, rd=1 (recursion desired)
128, # ra=1 (recursion available), z=0,
# rcode=0
0, 1, # Number of questions: 1
0, 1, # Number of answers: 1
0, 0, # Authority RRs: 0
0, 0, # Additional RRs: 0
--- QUESTION 1 of 1 ---
6, 103, 111, 111, 103, 108, 101,# Label "google"
3, 99, 111, 109, # Label "com"
0, # End of labels
0, 1, # Class "IN" (1)
0, 1 # Record type "A" (1)
--- ANSWER 1 of 1 ---
192, 12, # "google.com" by pointer to packet
# data at index 12 (question start)
0, 1, # Class "IN" (1)
0, 1, # Resource type "A" (1)
0, 0, 0, 182, # TTL=182
0, 4, 216, 58, 220, 142 # Address (len=4) [216.58.220.142]

Local Unbound 1.13.1 test server using HTTP/2:
https://unbound.example.org/dns-query?dns=OmYBAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ==

HTTP/2 Response Headers:
Content-Type: application/dns-message
Content-Length: 27

RAW REPLY, 27 BYTES:
58, 102, # Query ID (matches question ID)
129, # qr=1 (answer), opcode=0, aa=0,
# rc=0, rd=1 (recursion desired)
1, # ra=0 (recursion NOT available),
# z=0, rcode=1
# rcode=1 (format error) WHAT??
0, 1, # Number of questions: 1
0, 0, # Number of answers: 0
0, 0, # Authority RRs: 0
0, 0, # Additional RRs: 0
--- QUESTION 1 of 1 ---
6, 103, 111, 111, 103, 108, 101,# Label "google"
3, 99, 111, 109, # Label "com"
0, # End of labels
0, 1, # Class "IN" (1)
0, XXX MISSING BYTE XXX # Record type ??? LENGTH TOO SHORT!!

So now my questions.

1) WHY is Unbound NOT liking the question's format ("format error" as seen in rcode=1) when it IS in application/dns-message format, URL-safe base 64 encoded as part of the GET query?

2) WHY is Unbound's reply ONE BYTE short, and thus most DNS response parsers will raise an exception attempting to parse the reply? At first I thought perhaps the CURL library I was using to perform queries was truncating the reply, until I checked the HTTP/2 response headers and saw that the "Content-Length" header indeed contained the value 27.

Problem #2 looks like a BUG to me. As for #1... I have NO idea.

3) Can Unbound accept DoH queries via HTTP/1.1 or HTTP/1.0? When I attempt to query with these protocols, I don't even get a response at all. The TLS connection just gets closed. Cloudflare's server happily accepts HTTP/1.0 and HTTP/1.1 queries. I guess I can understand not supporting 1.0.

Thanks in advance for any and all answers, pointers, insights, or information regarding #1 and #2.

As for #3, it's not really too important to me. I assume the HTTP library Unbound is using doesn't have 1.1 support--and modern apps/systems really are most likely to do DoH with HTTP/2. I'd still like to see 1.1 supported someday, though.

Thanks, Unbound devs, for some excellent software!

--Aaron out

Hi,

I've been trying out DoH using Unbound 1.13.1 on a FreeBSD host and a Let's Encrypt TLS certificate. Unbound starts and listens on my DoH port, and when I connect to it, the TLS session is established as expected. I can send DNS queries and the server sends me a response, but it's one byte short and is simply a reply containing NO RR records, only the original question sent to the server, oddly truncated by a single byte.

For example, here's what happens when I query...<<snip>>

<<snip>>

Local Unbound 1.13.1 test server using HTTP/2:
https://unbound.example.org/dns-query?dns=OmYBAAABAAAAAAAABmdvb2dsZQNjb20AAAEAAQ==

<<snip>>

So now my questions.

1) WHY is Unbound NOT liking the question's format ("format error" as seen in rcode=1) when it IS in application/dns-message format, URL-safe base 64 encoded as part of the GET query?

I should add that when I attempt a non-dns-message style query to my server's "/dns-query" DoH endpoint, I simply get a 404 "Not Found" error message, again using HTTP/2, and including Accept: headers for whatever DoH reply type the server wants, application/dns-json, application/dns+json, or application/dns-message.

https://unbound.example.org/dns-query-foo?name=google.com&type=A

404 "Not Found"

I assume this query type isn't supported. Am I assuming foolishly and should I instead be looking for a configuration typo?

     tls\-service\-key: &quot;/foo/unbound/conf/cert\.key&quot;
     tls\-service\-pem: &quot;/foo/unbound/conf/cert\.pem&quot;
     \.\.\.
     http\-endpoint: &quot;/dns\-query&quot;

<<snip>>

Thanks, Unbound devs, for some excellent software!

--Aaron out

Thanks again!

--Aaron out

<<snip>>

I should add that when I attempt a non-dns-message style query to my server's "/dns-query" DoH endpoint, I simply get a 404 "Not Found" error message, again using HTTP/2, and including Accept: headers for whatever DoH reply type the server wants, application/dns-json, application/dns+json, or application/dns-message.

https://unbound.example.org/dns-query-foo?name=google.com&type=A

404 "Not Found"

The URL I tested was did NOT contain '-foo' as I managed to introduce that typo while editing my email message. Ooops, sorry! It should read:

https://unbound.example.org/dns-query?name=google.com&type=A

404 "Not Found"

Thanks!

--Aaron out

Hi,

you didn't describe, which client you used to send the DoH query.

Here are my favorites:

1)
As I compile unbound myself, I can run "make dohclient"
# dohclient
usage: dohclient [options] name type class ...
        sends the name-type-class queries over DNS-over-HTTPS.
-s server IP address to send the queries to, default: 127.0.0.1
-p Port to connect to, default: 443
-P Use POST method instead of default GET
-e HTTP endpoint, default: /dns-query
-c Content-type in request, default: application/dns-message
-n no-tls, TLS is disabled
-h This help text

2)
kdig: (maybe modern dig version also support DoH)
# kdig -p 53 hostname.bind. txt ch @127.0.0.1 +https
# kdig -p 53 hostname.bind. txt ch @127.0.0.1 +https-get

3)
a recent version of curl
# curl --verbose --doh-url https://unbound.example:443/dns-query https://nlnetlabs.nl

all versions work here with 1.13.1 on Debian/Linux

Andreas

Hi,

I've been trying out DoH using Unbound 1.13.1 on a FreeBSD host and a Let's Encrypt TLS certificate. Unbound starts and listens on my DoH port, and when I connect to it, the TLS session is established as expected. I can send DNS queries and the server sends me a response, but it's one byte short and is simply a reply containing NO RR records, only the original question sent to the server, oddly truncated by a single byte.

Hi,

you didn't describe, which client you used to send the DoH query.

I sent the HTTP/2 GET query using libcurl's facilities. I don't believe the querying code nor HTTP/2 HTTP/1.1 HTTP/1.0 implementation libcurl uses is related to why the server's response is truncated, one byte short of a valid application/dns-message response. Whether I send it using libcurl or from the CLI with the curl command directly, the issue is the same.

And to be clear, the client isn't doing the reply truncation. The HTTP/2 server response clearly includes a "Content-Length: 27" header, indicating the FULL reply is exactly 27 bytes in size. And I fully documented in my original post how it SHOULD have been a 28-byte reply to be a valid "Content-Type: application/dns-message" response. The "question" section of the response was one-byte short, supplying only a single zero byte where a two-byte value is described in the DNS RFCs for the question section's rtype field. Two bytes, a zero followed by a one, were expected, where only a single zero byte was provided. This is why I suspect this truncation issue might be a bug in Unbound.

Andreas

Thanks for your reply!

I didn't realize that curl's more recent incarnation added a --doh-url option. That might prove useful in the future.

--Aaron out.

as the DoH-implementation depends on libnghttp2: which version do you use?
I'm on 1.43.0 (https://packages.debian.org/bullseye/libnghttp2-14)

unforunately, "unbound -V" don't mention this version
in contrast to "Linked libs: libev 4.33 (it uses epoll), OpenSSL 1.1.1k 25 Mar 2021"

@nlnetlabs: feature request :slight_smile:

Andreas

ok, then maybe the developers @nlnetlabs may help ...

"A. Schulze via Unbound-users" writes:

>
> >> Andreas
> > Hmmm, I'm using the FreeBSD prebuilt package from their port system. Let me see if I can find what libnghttp2 version it was built with. Ah, libnghttp2-1.43.0 is the FreeBSD prebuilt package it installed as a dependency.
>
> ok, then maybe the developers @nlnetlabs may help ...

According to the RFC, https://datatracker.ietf.org/doc/html/rfc8484#section-5.2

  5.2. HTTP/2

     HTTP/2 [RFC7540] is the minimum RECOMMENDED version of HTTP for use
     with DoH.

     The messages in classic UDP-based DNS [RFC1035] are inherently
     unordered and have low overhead. A competitive HTTP transport needs
     to support reordering, parallelism, priority, and header compression
     to achieve similar performance. Those features were introduced to
     HTTP in HTTP/2 [RFC7540]. Earlier versions of HTTP are capable of
     conveying the semantic requirements of DoH but may result in very
     poor performance.

No point into not following this recommendation.

  jaap