Stop service.arpa requests being sent to upstream DNS

jon · 9 January 2025 20:28

I was searching through my message logs for SERVFAIL errors and I came across LOTS of these messages:

Oct  2 17:53:31 ipfire unbound: [15153:0] error: SERVFAIL <default.service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.112 got SERVFAIL
Oct  2 17:53:31 ipfire unbound: [15153:0] error: SERVFAIL <service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.11 got SERVFAIL
Oct  2 17:53:32 ipfire unbound: [15153:0] error: SERVFAIL <_matter._tcp.default.service.arpa. PTR IN>: all the configured stub or forward servers failed, at zone . from 9.9.9.11 got SERVFAIL
Oct  2 17:53:40 ipfire unbound: [15153:0] error: SERVFAIL <_L3887._sub._matterc._udp.default.service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.11 got SERVFAIL
Oct  2 17:53:41 ipfire unbound: [15153:0] error: SERVFAIL <_sub._matterc._udp.default.service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 9.9.9.11 got SERVFAIL
Oct  2 17:53:41 ipfire unbound: [15153:0] error: SERVFAIL <_matterc._udp.default.service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.112 got SERVFAIL
Oct  2 17:53:42 ipfire unbound: [15153:0] error: SERVFAIL <_udp.default.service.arpa. SOA IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.11 got SERVFAIL
Oct  2 17:53:42 ipfire unbound: [15153:0] error: SERVFAIL <_L3887._sub._matterc._udp.default.service.arpa. PTR IN>: all the configured stub or forward servers failed, at zone . from 9.9.9.11 got SERVFAIL
Oct  2 17:53:52 ipfire unbound: [15153:0] error: SERVFAIL <1234567890123456-1234567890123456._matter._tcp.default.service.arpa. SRV IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.112 got SERVFAIL
Oct  2 17:53:52 ipfire unbound: [15153:0] error: SERVFAIL <1234567890123456-1234567890123456._matter._tcp.default.service.arpa. TXT IN>: all the configured stub or forward servers failed, at zone . from 149.112.112.112 got SERVFAIL

I believe these are all local DNS lookup and are failing an external DNS lookup for service.arpa.

All of the above messages started on Oct 2, 2024. At that time I was using:

CU 188
unbound 1.21.0
APU4D4

There are 1000s of these messages SERVFAIL . . . service.arpa every week and 144028 since Oct 2024.

These message still persist today.

I Giggled around for service.arpa and found this:

you may have missed that, but IANA decided to properly delegate previously to home.arpa - to avoid DNSSEC breakage.

from here:

I have no idea if the two are related or how they are related.

Is there an unbound (or maybe bind) way to block service.arpa dns requests to the outside world. Temporarily I did block them via RPZ, but that seems doesn’t feel like the right way to fix this.

EDIT: added link

Other link:

bonnietwin · 10 January 2025 11:07

I have checked my logs from start of Nov 2024 to today and I don’t have a single service.arpa entry in my unbound logs on my production server.

If you believe that these are supposed to be local DNS lookups then it seems to me that the only reason for these to be sent external would be if they could not be resolved locally (bear in mind I am also not an expert).

Do you have all your hostnames on your local domain listed and enabled in the Hosts WUI page. I would expect any local fqdn listed in the Hosts page should be able to get resolved locally.

jon · 10 January 2025 15:15

This helps! I have mostly Apple related devices on my network.

Do you have any arpa escaping to upstream (external) DNS?

zgrep -c -ai "SERVFAIL.*arpa" $(ls -1tr /var/log/messages* | tail -12)

I am guessing not, but I wanted to be sure.

I would think anything “local” would always stay local. Me fat fingering a hostname doesn’t cause an outside lookup.

[root@ipfire ~] # ping mm2
PING mm2.localdomain (192.168.60.20) 56(84) bytes of data.
64 bytes from MM2.localdomain (192.168.60.20): icmp_seq=1 ttl=64 time=0.615 ms
64 bytes from MM2.localdomain (192.168.60.20): icmp_seq=2 ttl=64 time=0.585 ms
^C
--- mm2.localdomain ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.585/0.600/0.615/0.015 ms

[root@ipfire ~] # ping mm2a
ping: mm2a: Name or service not known
[root@ipfire ~] #

None that end in arpa or service.arpa. They are all simple hostnames.

This is a wild guess here and I may be going down a rabbit hole . . .

Since many of the SERVFAIL items include matter I am wondering if it is related to this:

I do not have any devices (that I know of) that are matter-based.

EDIT: Per ChatGPT the answer is “yes”.

But this Rabbit Hole doesn’t explain why service.arpa. makes it to an outside DNS.

jon · 10 January 2025 16:00

Wondering out loud . . .

Maybe there needs to be a local zone for service.arpa?

There is one for home.arpa

[root@ipfire ~] # unbound-control list_local_zones
0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa. static
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa. static
8.b.d.0.1.0.0.2.ip6.arpa. static
d.f.ip6.arpa. static
8.e.f.ip6.arpa. static
9.e.f.ip6.arpa. static
a.e.f.ip6.arpa. static
b.e.f.ip6.arpa. static
home.arpa. static
0.in-addr.arpa. static
10.in-addr.arpa. static
...

bonnietwin · 10 January 2025 17:03

Okay, I have no Apple related devices at all. Mine are all Linux based except for my phone which is Android.

Then I have a range of things like TV, Surround Sound system, that are connected into my Orange Zone.

I ran your command and it came back with small numbers such as 0 to 6 at the max.

There were two with 6, these covering the second half of November.

I then looked in the whole of November again and there were 16 entries that had servfail <IP.in-addr.arpa. these all being when my DDNS linkage to my Public IP got lost for a short while.

The messages were either “misc failure” or “exceeded the maximum nameserver nxdomains”

Sounds reasonable to me. Maybe you could test it out and see if it helps. If it stops your messages. However not sure how to get it added to the local-zone list. There doesn’t appear to be any default local-zone list that is defined for IPFire so maybe just the default one from unbound is used.

EDIT: unbound must somehow get some of the information itself from the used network as my domain name used internally is also listed in the local-zone.

jon · 10 January 2025 17:56

It seems to work with initial tests. I’ll let it run for a day to double-triple check.

I create a file at /etc/unbound/local.d/service-arpa.conf

[root@ipfire ~] # cat /etc/unbound/local.d/service-arpa.conf
server:
    local-zone: "service.arpa." static
[root@ipfire ~] #

and then did an /etc/rc.d/init.d/unbound restart

and I made sure service.arpa was not blocked by RPZ.

Same for me. I tried looking for:

home.arpa
onion
invalid

These are all included in unbound-control list_local_zones and in unbound-control list_local_data and they come from somewhere!

Maybe from an external link that is included in the build??

It “feels” like it should be part of unbound code and maybe created in localzone.c

github.com

NLnetLabs/unbound/blob/master/services/localzone.c#L941-L971


      
          	/* home.arpa. zone (RFC 8375) */
          	if(!add_empty_default(zones, cfg, "home.arpa.")) {
          		log_err("out of memory adding default zone");
          		return 0;
          	}
          	/* onion. zone (RFC 7686) */
          	if(!add_empty_default(zones, cfg, "onion.")) {
          		log_err("out of memory adding default zone");
          		return 0;
          	}
          	/* test. zone (RFC 6761) */
          	if(!add_empty_default(zones, cfg, "test.")) {
          		log_err("out of memory adding default zone");
          		return 0;
          	}
          	/* invalid. zone (RFC 6761) */
          	if(!add_empty_default(zones, cfg, "invalid.")) {
          		log_err("out of memory adding default zone");
          		return 0;
          	}

This file has been truncated. show original

In this document search for The default zones are localhost or home.arpa (RFC 8375)

bonnietwin · 10 January 2025 19:46

That suggests that there is a default set that is defined by unbound and that includes home.arpa. So if service.arpa has been added by IANA now then it looks like we will have to wait for unbound to update their default local zone set to include service.arpa.

home.arpa was added to unbound in version 1.14.0 from Dec 2021.

RFC8375 for home.arpa was released in May 2018.

At the moment there is no RFC for service.arpa. There is a draft document that mentions that service.arpa should be dealt with in the same way as home.arpa, which was released in Sep 2024.
https://datatracker.ietf.org/doc/html/draft-ietf-dnssd-srp-25#name-delegation-of-servicearpa

EDIT: A pdf file for RFC9665 looks to be an officially released RFC for this issue. However I haven’t been able to find the actual RFC on the ietf website. The pdf is dated Oct 2024.
https://ftp.nic.ad.jp/in-notes/authors/rfc9665.pdf

So it could be a few years still before service.arpa gets added into the local-zone.

So you might need to leave that entry in your unbound local.d directory.

Don’t forget to add it to your include.user file so it gets backed up by IPFire so you don’t miss it when you do a restore after a fresh install.

EDIT: The two authors of that draft proposal both work for Apple Inc.

jon · 10 January 2025 20:13

If others find these “SERVFAIL . . . service.arpa” errors in their logs, we can always add this quick fix to the standard build for the short term until it appears in the unbound code.

Thanks for your help!

bbitsch · 10 January 2025 20:52

I didn’t find any occurence of service.arpa, whether with SERVFAIL nor without.

jon · 10 January 2025 22:22

Hi Bernhard - do you have any Apple related devices n your network?

bbitsch · 11 January 2025 09:48

Short answer: No. And no IoT devices either.

jon · 11 January 2025 18:26

Jon:

It seems to work with initial tests. I’ll let it run for a day to double-triple check.

I create a file at /etc/unbound/local.d/service-arpa.conf
[root@ipfire ~] # cat /etc/unbound/local.d/service-arpa.conf
server:
    local-zone: "service.arpa." static
[root@ipfire ~] # 
and then did an /etc/rc.d/init.d/unbound restart

After one day, so far so good! No service.arpa requests being sent to upstream DNS. And no other odd issues found.

jon · 14 January 2025 17:05

I sent a note to the unbound group and it looks like there will be a future update for service.arpa (and for resolver.arpa).

Here is the beginning of the thread:
https://lists.nlnetlabs.nl/pipermail/unbound-users/2025-January/008465.html

click Next message (by thread) to see other responses.

jon · 14 January 2025 19:15

I went through the message logs and searched for resolver.arpa and I was getting 1000’s of these messages also:

Jan 14 11:03:01 ipfire unbound: [28585:0] info: validation failure <_dns.resolver.arpa. SVCB IN>: no NSEC3 records from 9.9.9.11 for DS resolver.arpa. while building chain of trust
Jan 14 11:03:01 ipfire unbound: [28585:0] info: validation failure <_dns.resolver.arpa. SVCB IN>: key for validation resolver.arpa. is marked as invalid because of a previous
Jan 14 11:05:58 ipfire unbound: [28585:0] info: validation failure <_dns.resolver.arpa. SVCB IN>: no NSEC3 records from 149.112.112.112 for DS resolver.arpa. while building chain of trust
Jan 14 11:11:57 ipfire unbound: [28585:0] info: validation failure <_dns.resolver.arpa. SVCB IN>: no NSEC3 records from 9.9.9.11 for DS resolver.arpa. while building chain of trust

Since they were an unbound: [PID] info: message, and considered INFORMATION, and not a unbound: [PID] error: SERVFAIL I was ignoring them.

I was getting 1000s of the resolver.arpa messages every week for the past year.

[root@ipfire /] # zgrep -c -ai "resolver.arpa" $(ls -1tr /var/log/messages* | tail -52)
/var/log/messages.51.gz:3424
/var/log/messages.50.gz:3558
/var/log/messages.49.gz:3414
/var/log/messages.48.gz:3446
. . .
/var/log/messages.3.gz:4450
/var/log/messages.2.gz:4078
/var/log/messages.1.gz:4207
/var/log/messages:1392
[root@ipfire /] #

I added resolver.arpa. like this:

[root@ipfire /] # cat /etc/unbound/local.d/service-arpa.conf
server:
    local-zone: "service.arpa." static
    local-zone: "resolver.arpa." static
[root@ipfire /] #

And poof they no longer appear in the messages log.

Ahhhhh! My logs seem so much lighter!

gw-ipfire · 15 January 2025 14:24

hi
i have see this resolver.arpa in log a unbond i block But I don’t know what it’s for.

jon · 15 January 2025 18:26

I am not sure either. I found this:

and this:

Identifying a DNS resolver:

If you see a DNS query for “resolver.arpa,” it means that a device is trying to determine the domain name associated with the IP address of the DNS server it is currently using.

And I can see them in my logs for the past year.

EDIT:
And this is interesting (though I haven’t completed the thread yet)