(apologies for not responding to the original, but it's gone from my inbox)
Before I go any further, this is not a DNS issue or an issue with unbound. This is an issue with nginx setup and understanding TLS.
I do this with nginx, although not targeting unbound. Originally I did it following the instructions included with BIND, and although the latest versions of BIND support DoT I continue to use it with other DNS-based services.
Before you do anything else:
* make sure that your unbound instance will accept a TCP connection on port 53
Now we have confidence that if it gets a valid TCP stream it will respond accordingly. So let's debug that tunnel.
1) open two terminal windows
2) to create a listener in one, type:
nc -lk <local-address> <local-port>
3) in the other send it a message:
nc <listener-address> <listener-port>
test!
^C
That should work. Ok? Now jack nginx in there sideways across all four lanes of the highway and test again:
4) in your custom (for testing) nginx config:
stream {
server {
listen <listener-address>:<ssl-port> ssl;
proxy_pass <listener-address>:<listener-port>;
5) send it a message, using openssl instead of nc in step 3:
openssl s_client -quiet -connect <listener-address>:<ssl-port>
[some debug information is printed]
hello!
^C
If that doesn't work, then fix it.
Now you know that you have a good tunnel.
[...]
My guess is you need to put proxy_protocol on; in upstream dns to tell nginx to talk to it's backend.
Although I think this is a misconception and that's why I'm responding...
Hello, I was trying to set up a DoT -> nginx -> unbound scheme but
At first blush "DoT -> nginx -> unbound" sounds sensible, but what exactly does it mean?
Let's try this instead:
TLS(TCP(DNS)) --TLS-> nginx --TCP-> unbound
0) "DNS" is a DNS message in the format which would be sent via UDP.
1) The DNS message is prepended with a length word and encapsulated in a
TCP stream, which is then encapsulated in a TLS session.
2) This is tunnelled via TLS to nginx.
3) nginx provides the other end of the tunnel.
4) The TCP stream surfaced by nginx is sent to unbound.
Normally there will be one nginx at the server end and in close proximity to it. Nginx is not being used at the client end of the tunnel.
What should be going into that tunnel from the client end is a legitimate DNS TCP stream (the output of step 1).
[...]
unbound log:
error: proxy_protocol: could not parse PROXYv2 header
nginx log:
SSL_shutdown() failed (SSL: error:14094123:SSL
routines:ssl3_read_bytes:application data after close notify) while
proxying connection, client: <client_ipv4>, server: <server_ipv4>:853, upstream: "[::1]:853", bytes from/to client:0/0, bytes from/to 0/0
You need something to create a TLS session. If unbound is complaining about e.g. proxy_protocol then that's cruft which needs to be eliminated.
You need a transparent TLS tunnel within which to transport DNS over TCP.
Unbound is not seeing correct output, which should be DNS in a TCP stream:
* nginx is mistakenly sending this as some kind of handshake to negotiate
something (there is nothing to negotiate!)
* whatever was fed into the nginx tunnel at the client end is not DNS in a
TCP stream
Nginx is having issues negotiating a TLS session:
* that is not really nginx at the server end of the tunnel
* nginx (at the server end) is not properly configured to terminate the
TLS tunnel
If you want an example of programmatically taking a UDP DNS request, encapsulating that in a TCP stream and putting that in a TLS tunnel, see:
https://github.com/m3047/tcp_only_forwarder