Local DNS problems after update to core 144

Hello,

after the core 144 update I notice regular problems with resolving local DNS names (or similar).

I have configured some hosts in ipfire, e.g. server1.local_domain. Several times a day my browser tells me “website not found” when trying to connect to server1.local_domain, ping server1.local_domain tells “name or service not found”. There are more services having this problem, some do not.

  • server1.local_domain does not work but e.g. server2.local_domain, but maybe server2.local_domain will not work later, too
  • Edit: it could be that connection to server1.local_domain works from client 1 but not from client 2
  • There is no problem with domains in the internet
  • This problem was noticed for clients in my local network and for mobile devices via OpenVPN
  • This state remains for a while, maybe it works again later
  • I can force it working again when disconnection the client device from network and connect it again

Does anyone see similar problems? I have no idea what the problem could be. I scanned manually the logs but can’t find any interesting entries…

Regards

Hi,

there currently is a patch in the queue for
the next Core Update to change Unbounds resolution behaviour on locally configured hosts.
Perhaps this will fix your problem.

Meanwhile, what does a dig a server1.local_domain return if things do not work?
NXDOMAIN or SERVFAIL?

Could you post all log lines in /var/log/messages containing unbound here, please?
In case of SERVFAILs, it should log there why a query could not be answered.

Thanks, and best regards,
Peter Müller

Hello Peter,

thanks for your answer. Here is the first part:

 <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> wiki.LOCAL_DOMAIN
 ;; global options: +cmd
 ;; Got answer:
 ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 53979
 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
 
 ;; OPT PSEUDOSECTION:
 ; EDNS: version: 0, flags:; udp: 65494
 ;; QUESTION SECTION:
 ;wiki.LOCAL_DOMAIN.			IN	A
 
 ;; Query time: 0 msec
 ;; SERVER: 127.0.0.53#53(127.0.0.53)
 ;; WHEN: Thu Apr 30 12:05:49 CEST 2020
 ;; MSG SIZE  rcvd: 39

Hello again,
and here some unbound logs, no SERVFAIL found.

Apr 29 01:21:21 ipfire unbound: [7933:0] info: service stopped (unbound 1.10.0).
Apr 29 01:21:21 ipfire unbound: [7933:0] info: server stats for thread 0: 2932 queries, 959 answers from cache, 1973 recursions, 66 prefetch, 0 rejected by ip ratelimiting
Apr 29 01:21:21 ipfire unbound: [7933:0] info: server stats for thread 0: requestlist max 16 avg 2.01962 exceeded 0 jostled 0
Apr 29 01:21:21 ipfire unbound: [7933:0] info: average recursion processing time 0.671964 sec
Apr 29 01:21:21 ipfire unbound: [7933:0] info: histogram of recursion processing times
Apr 29 01:21:21 ipfire unbound: [7933:0] info: [25%]=0.184073 median[50%]=0.397098 [75%]=0.788949
Apr 29 01:21:21 ipfire unbound: [7933:0] info: lower(secs) upper(secs) recursions
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.000000    0.000001 295
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.002048    0.004096 1
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.004096    0.008192 3
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.008192    0.016384 2
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.016384    0.032768 1
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.032768    0.065536 5
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.065536    0.131072 52
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.131072    0.262144 332
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.262144    0.524288 574
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    0.524288    1.000000 386
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    1.000000    2.000000 153
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    2.000000    4.000000 143
Apr 29 01:21:21 ipfire unbound: [7933:0] info:    4.000000    8.000000 26
Apr 29 01:21:31 ipfire unbound: [22374:0] notice: init module 0: validator
Apr 29 01:21:31 ipfire unbound: [22374:0] notice: init module 1: iterator
Apr 29 01:21:32 ipfire unbound: [22374:0] info: start of service (unbound 1.10.0).
Apr 29 01:25:01 ipfire unbound: [22374:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 29 09:18:52 ipfire unbound: [22374:0] info: validation failure <local. SOA IN>: no NSEC3 records from 80.241.218.68 for DS local. while building chain of trust
Apr 29 13:10:57 ipfire unbound: [22374:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 29 13:50:28 ipfire unbound: [22374:0] info: validation failure <local. SOA IN>: no DNSSEC records from 176.9.93.198 for DS local. while building chain of trust
Apr 29 14:40:17 ipfire unbound: [22374:0] info: service stopped (unbound 1.10.0).
Apr 29 14:40:17 ipfire unbound: [22374:0] info: server stats for thread 0: 2889 queries, 1040 answers from cache, 1849 recursions, 73 prefetch, 0 rejected by ip ratelimiting
Apr 29 14:40:17 ipfire unbound: [22374:0] info: server stats for thread 0: requestlist max 13 avg 1.02029 exceeded 0 jostled 0
Apr 29 14:40:17 ipfire unbound: [22374:0] info: average recursion processing time 0.609996 sec
Apr 29 14:40:17 ipfire unbound: [22374:0] info: histogram of recursion processing times
Apr 29 14:40:17 ipfire unbound: [22374:0] info: [25%]=0.192377 median[50%]=0.351745 [75%]=0.541704
Apr 29 14:40:17 ipfire unbound: [22374:0] info: lower(secs) upper(secs) recursions
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.000000    0.000001 150
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.032768    0.065536 4
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.065536    0.131072 109
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.131072    0.262144 426
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.262144    0.524288 689
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    0.524288    1.000000 239
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    1.000000    2.000000 78
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    2.000000    4.000000 132
Apr 29 14:40:17 ipfire unbound: [22374:0] info:    4.000000    8.000000 22
Apr 29 14:40:17 ipfire unbound: [22374:0] notice: Restart of unbound 1.10.0.
Apr 29 14:40:25 ipfire unbound: [22374:0] notice: init module 0: validator
Apr 29 14:40:25 ipfire unbound: [22374:0] notice: init module 1: iterator
Apr 29 14:40:25 ipfire unbound: [22374:0] info: start of service (unbound 1.10.0).
Apr 29 14:40:25 ipfire unbound: [22374:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 29 14:50:22 ipfire unbound: [22374:0] info: service stopped (unbound 1.10.0).
Apr 29 14:50:22 ipfire unbound: [22374:0] info: server stats for thread 0: 23 queries, 3 answers from cache, 20 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 29 14:50:22 ipfire unbound: [22374:0] info: server stats for thread 0: requestlist max 3 avg 0.65 exceeded 0 jostled 0
Apr 29 14:50:22 ipfire unbound: [22374:0] info: average recursion processing time 0.837544 sec
Apr 29 14:50:22 ipfire unbound: [22374:0] info: histogram of recursion processing times
Apr 29 14:50:22 ipfire unbound: [22374:0] info: [25%]=0.229376 median[50%]=0.471859 [75%]=0.904858
Apr 29 14:50:22 ipfire unbound: [22374:0] info: lower(secs) upper(secs) recursions
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    0.032768    0.065536 2
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    0.131072    0.262144 4
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    0.262144    0.524288 5
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    0.524288    1.000000 5
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    1.000000    2.000000 2
Apr 29 14:50:22 ipfire unbound: [22374:0] info:    2.000000    4.000000 2
Apr 29 14:54:01 ipfire unbound: [19950:0] notice: init module 0: validator
Apr 29 14:54:01 ipfire unbound: [19950:0] notice: init module 1: iterator
Apr 29 14:54:01 ipfire unbound: [19950:0] info: start of service (unbound 1.10.0).
Apr 29 14:54:59 ipfire unbound: [19950:0] info: service stopped (unbound 1.10.0).
Apr 29 14:54:59 ipfire unbound: [19950:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Apr 29 14:54:59 ipfire unbound: [19950:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Apr 29 14:54:59 ipfire unbound: [19950:0] notice: Restart of unbound 1.10.0.
Apr 29 14:55:07 ipfire unbound: [19950:0] notice: init module 0: validator
Apr 29 14:55:07 ipfire unbound: [19950:0] notice: init module 1: iterator
Apr 29 14:55:07 ipfire unbound: [19950:0] info: start of service (unbound 1.10.0).
Apr 29 14:55:08 ipfire unbound: [19950:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 29 15:50:36 ipfire unbound: [19950:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 30 01:21:21 ipfire unbound: [19950:0] info: service stopped (unbound 1.10.0).
Apr 30 01:21:21 ipfire unbound: [19950:0] info: server stats for thread 0: 4552 queries, 1338 answers from cache, 3214 recursions, 73 prefetch, 0 rejected by ip ratelimiting
Apr 30 01:21:21 ipfire unbound: [19950:0] info: server stats for thread 0: requestlist max 20 avg 1.68969 exceeded 0 jostled 0
Apr 30 01:21:21 ipfire unbound: [19950:0] info: average recursion processing time 0.558780 sec
Apr 30 01:21:21 ipfire unbound: [19950:0] info: histogram of recursion processing times
Apr 30 01:21:21 ipfire unbound: [19950:0] info: [25%]=0.12664 median[50%]=0.259693 [75%]=0.487063
Apr 30 01:21:21 ipfire unbound: [19950:0] info: lower(secs) upper(secs) recursions
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.000000    0.000001 555
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.008192    0.016384 2
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.016384    0.032768 11
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.032768    0.065536 8
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.065536    0.131072 244
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.131072    0.262144 802
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.262144    0.524288 919
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    0.524288    1.000000 384
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    1.000000    2.000000 98
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    2.000000    4.000000 121
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    4.000000    8.000000 45
Apr 30 01:21:21 ipfire unbound: [19950:0] info:    8.000000   16.000000 24
Apr 30 01:21:21 ipfire unbound: [19950:0] info:   16.000000   32.000000 1
Apr 30 01:21:32 ipfire unbound: [16044:0] notice: init module 0: validator
Apr 30 01:21:32 ipfire unbound: [16044:0] notice: init module 1: iterator
Apr 30 01:21:32 ipfire unbound: [16044:0] info: start of service (unbound 1.10.0).
Apr 30 01:25:02 ipfire unbound: [16044:0] info: generate keytag query _ta-4a5c-4f66. NULL IN
Apr 30 11:41:30 ipfire unbound: [16044:0] info: generate keytag query _ta-4a5c-4f66. NULL IN

Hi,

these log lines are most likely caused by SERVFAILs:

Apr 29 09:18:52 ipfire unbound: [22374:0] info: validation failure <local. SOA IN>: no NSEC3 records from 80.241.218.68 for DS local. while building chain of trust
Apr 29 13:50:28 ipfire unbound: [22374:0] info: validation failure <local. SOA IN>: no DNSSEC records from 176.9.93.198 for DS local. while building chain of trust

Since .local is reserved for multicast DNS, I guess it’s best to switch to a
different internal domain as leaking mDNS queries to DNS is neither robust nor
desirable. :slight_smile:

Thanks, and best regards,
Peter Müller

Hi,

if we were to use a different internal domain how are we supposed to teach unbound to use it? After I had found the following post from the old forum:

https://forum.ipfire.org/viewtopic.php?t=18680

I had a look in /etc/unbound/unbound.conf (Core 144) and did not find a clue. This wiki entry does not help:

https://wiki.ipfire.org/dns/dnssec/whitelisting

Meanwhile the folder and file structure of unbound seem to have been changed. “INSECURE_ZONES =” as in the linked thread seems not to be used anymore.

So how are we supposed to tell unbound to use a ‘private TLD’ as say for exampe ‘.intranet’?

Cheers

Gremlin

Hello Peter,

my local domain isn’t .local, it’s .famalyname .

I am having the same problem since the upgrade.

Here’s an excerpt from my settings.

I have watch on dig set for one of them and every few minutes it fails to resolve. Then, after a few minutes, it resolves correctly again for a while.

Below is an excerpt of a packet trace.

You will notice that the query that succeeds is an IPv4 query, while the one that fails is an IPv6 query.

I am not sure what has changed, yet. I have not made changes to my host (Ubuntu 19.10), but an update may have.

It seems, however, that Unbound does not respond correctly to the AAAA request and instead of returning what is in the hosts settings, it returns the results from the authoritative nameserver on the internet.

Hi,

I am pretty sure https://patchwork.ipfire.org/patch/3018/ fixes your problem,
but to ensure it does, could you test this patch and report back please?

Thanks, and best regards,
Peter Müller

2 Likes

I would, if I knew how to apply the patch. It looks like it is a patch for the IpFire build system.

Maybe you could tell me what to change in which file(s) on my IpFire?

I am still not sure why this started at all. I have been tracking two of my systems for a while now. One, Ubuntu 20.04, has not sent an AAAA query once, while my Ubuntu 19,10 machine has several times. Even though I have disabled IPv6 entirely!

Found the file, I think. /etc/rc.d/init.d/unbound, right?

I changed that, but it did not help. I still get an unknown address response, now from Unbound itself, instead of the external name server, it seams.

The whole problem seems to be that my system asks for an IPv6 address every once in a while. I have no idea why. I don’t know if it did before, or if it didn’t, when it started.

I have not changed my network settings. Maybe it will go away when I upgrade to Ubuntu 20.04 when the .1 release is here.

I do also have Ubuntu / Linux Mint clients in my network. All have that problem. I noticed that my Androids do not have the problems (yet).

Hi,

this sounds like Ubuntu asks for an IPv6 address first, and falls back to IPv4 if
the query did not return any sufficient response. Since there is only an IPv4 address
configured locally, the other query will be emitted to an external resolver.

Even when using core utilities like host, an IPv6 DNS query is being made:

user@machine:~> host example.com
example.com has address 93.184.216.34
example.com has IPv6 address 2606:2800:220:1:248:1893:25c8:1946
example.com mail is handled by 0 .

There are two problems about this:

  1. We cannot predict the applications’ behaviour. Some applications might ask
    for an IPv6 address first, some do not, some query for v4 and v6 in parallel.

  2. In your case, you probably want any query to a locally configured host to
    return NXDOMAIN, except for IPv4 queries. Some other people apparently need
    to forward the former to external resolvers, which breaks you setup and vice
    versa.

I have no good idea about what to on this. Adding another switch somewhere in the
WUI would make things more complicated and confusing.

@ms: Thoughts?

Thanks, and best regards,
Peter Müller

Unbound is just handling this awfully wrong here. I assumed that it would do something else from what the documentation says.

When the system queries for a name and there is no AAAA record, it cannot return NXDOMAIN. That would mean that nothing exists under that name which is not true. In this example, an A record exists, but the client’s DNS resolver has been told that nothing exists and therefore it is not trying any more.

So, the client is behaving correctly. Unbound isn’t. The patch will fix it. We need to get it into the next update.

Hello @ms and @pmueller ,

thanks for your help. I patched the file and the problems are gone.

Thanks again!

Maybe this patch will ease so many DNS problems? Who knows… :slight_smile:

I happened to have the same problem with the second and third response showing “NXDOMAIN” while resolving internal hostnames in core 147.

I am pretty sure https://patchwork.ipfire.org/patch/3018/ fixes your problem,

Your patch of /etc/init.d/unbound (line 81) solved my problem 100%. Thanks!