Odd NTP Offset Issues (continued)

jon · 3 December 2019 17:06

Continued from old IPFire forum:

EDIT: this is an bad link to the old IPFire Forum:
Odd NTP Offset Issues
https://forum.ipfire.org/viewtopic.php?f=27&t=21861

H&M - I see many of the same messages in /var/log/messages:

Nov 29 15:07:27 ipfire ntpd[3258]: ntpd 4.2.8p13@1.3847-o Thu May  2 20:06:35 UTC 2019 (1): Starting
Nov 29 15:07:27 ipfire ntpd[3258]: Command line: /usr/bin/ntpd -Ap /var/run/ntpd.pid
Nov 29 15:07:27 ipfire ntpd[3260]: proto: precision = 0.240 usec (-22)
Nov 29 15:07:27 ipfire ntpd[3260]: basedate set to 2019-04-20
Nov 29 15:07:27 ipfire ntpd[3260]: gps base set to 2019-04-21 (week 2050)
Nov 29 15:07:27 ipfire ntpd[3260]: restrict default: KOD does nothing without LIMITED.
Nov 29 15:07:27 ipfire ntpd[3260]: Listen and drop on 0 v6wildcard [::]:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen and drop on 1 v4wildcard 0.0.0.0:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen normally on 2 lo 127.0.0.1:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen normally on 3 green0 192.168.xxx.xx:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen normally on 4 orange0 10.xxx.xxx.xx:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen normally on 5 red0 xxx.xxx.xxx.xx:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listen normally on 6 tun0 xx.xx.xxx.xx:123
Nov 29 15:07:27 ipfire ntpd[3260]: Listening on routing socket on fd #23 for interface updates
Nov 29 15:07:27 ipfire ntpd[3260]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Nov 29 15:07:27 ipfire ntpd[3260]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized

but not the last two lines:

Dec  3 08:25:09 silver-x86-64 ntpdate[19953]: no server suitable for synchronization found
Dec  3 08:25:09 silver-x86-64 ipfire: ntpdate error

What servers did you add to the “/etc/ntp.conf” file? Maybe the are not NTP-ing?

EDIT: Try this command with your NTP server:

[root@ipfire ~]# ntpdate -qu 0.ipfire.pool.ntp.org

server 159.203.158.197, stratum 2, offset 0.001489, delay 0.05814
server 195.21.137.209, stratum 2, offset -0.002432, delay 0.05913
server 216.229.0.50, stratum 2, offset -0.008718, delay 0.07213
server 69.164.213.136, stratum 2, offset -0.000237, delay 0.05652
 3 Dec 11:13:00 ntpdate[13585]: adjust time server 69.164.213.136 offset -0.000237 sec

Odd NTP Offset Issues - forum.ipfire.org.pdf.zip (272.2 KB)

hjkl · 3 December 2019 19:53

Hi,

I used in ntp.conf a bunch of *.pool.ntp.org servers.
I tested all of them manually with ntpdate -qu and all worked…manually!

Here is the output for ipfire ntp server:

ntpdate -qu 0.ipfire.pool.ntp.org
server 80.74.64.1, stratum 3, offset -0.001843, delay 0.07430
server 109.74.206.120, stratum 2, offset -0.001321, delay 0.06757
server 81.209.183.113, stratum 2, offset 0.000936, delay 0.08987
server 91.198.10.4, stratum 2, offset 0.007249, delay 0.08281
3 Dec 21:51:00 ntpdate[19433]: adjust time server 109.74.206.120 offset -0.001321 sec

jon · 3 December 2019 20:18

Something must be working since the offset is very small.

Set the Synchronization to Manually (menu Services > Time Server):

and take a look (or keep an eye) on the log.

hjkl · 3 December 2019 20:22

I did put the setup on Manual (as old forum say)
Good news: Drift is now populated!

cat /etc/ntp/drift
19.149

Log on the other hand is a mess: more than 850 lines like these:

Dec 3 22:20:09 silver-x86-64 ntpdate[20598]: no server suitable for synchronization found
Dec 3 22:20:09 silver-x86-64 ipfire: ntpdate error

grep ntpdate /var/log/messages |grep error |wc -l
847

Something is broken in my config if i get so many ntpdate error…

jon · 3 December 2019 20:32

I’ve not seen this error before. So let’s go into ~~guessing mode~~ troubleshooting mode. Try & post the results to this command:

ntpdate -dv 0.ipfire.pool.ntp.org

EDIT: Curious if you tried a IPFire box reboot?

hjkl · 3 December 2019 20:37

I found the “source” for all errors:

fcrontab contains this:

Set time

*/5 * * * * /usr/local/bin/timecheck > /dev/null 2>&1

If I run /usr/local/bin/timecheck I get the 2 ntpd lines in /var/log/messages:

Dec 3 22:33:35 silver-x86-64 ntpdate[21119]: no server suitable for synchronization found
Dec 3 22:33:36 silver-x86-64 ipfire: ntpdate error

So the error lines are generated by timecheck. 2 log lines every 5 minutes!

-dv output:

ntpdate -dv 0.ipfire.pool.ntp.org
3 Dec 22:36:11 ntpdate[21274]: ntpdate 4.2.8p13@1.3847-o Thu May 2 20:06:36 UTC 2019 (1)
Looking for host 0.ipfire.pool.ntp.org and service ntp
194.25.134.196 reversed to ntp1.sul.t-online.de
host found : ntp1.sul.t-online.de
transmit(194.25.134.196)
receive(194.25.134.196)
transmit(213.209.109.44)
receive(213.209.109.44)
transmit(5.189.146.13)
receive(5.189.146.13)
transmit(5.9.113.140)
receive(5.9.113.140)
transmit(194.25.134.196)
receive(194.25.134.196)
transmit(213.209.109.44)
receive(213.209.109.44)
transmit(5.189.146.13)
receive(5.189.146.13)
transmit(5.9.113.140)
receive(5.9.113.140)
transmit(194.25.134.196)
receive(194.25.134.196)
transmit(213.209.109.44)
receive(213.209.109.44)
transmit(5.189.146.13)
receive(5.189.146.13)
transmit(5.9.113.140)
receive(5.9.113.140)
transmit(194.25.134.196)
receive(194.25.134.196)
transmit(213.209.109.44)
receive(213.209.109.44)
transmit(5.189.146.13)
receive(5.189.146.13)
transmit(5.9.113.140)
receive(5.9.113.140)

server 194.25.134.196, port 123
stratum 2, precision -24, leap 00, trust 000
refid [172.20.96.197], root delay 0.001755, root dispersion 0.027740
reference time: e19143e3.d6478247 Tue, Dec 3 2019 22:28:19.837
originate timestamp: e19145c1.e2051460 Tue, Dec 3 2019 22:36:17.882
transmit timestamp: e19145c1.de17ac3b Tue, Dec 3 2019 22:36:17.867
filter delay: 0.06097 0.06125 0.06085 0.06117
---- ---- ---- ----
filter offset: -0.002538 -0.002397 -0.002560 -0.002507
---- ---- ---- ----
delay 0.06085, dispersion 0.00002, offset -0.002560

server 213.209.109.44, port 123
stratum 2, precision -24, leap 00, trust 000
refid [10.129.40.211], root delay 0.000259, root dispersion 0.008957
reference time: e19144a2.6df7b482 Tue, Dec 3 2019 22:31:30.429
originate timestamp: e19145c2.163747f7 Tue, Dec 3 2019 22:36:18.086
transmit timestamp: e19145c2.11534203 Tue, Dec 3 2019 22:36:18.067
filter delay: 0.06625 0.06630 0.06596 0.06625
---- ---- ---- ----
filter offset: -0.001265 -0.001346 -0.001367 -0.001288
---- ---- ---- ----
delay 0.06596, dispersion 0.00006, offset -0.001367

server 5.189.146.13, port 123
stratum 2, precision -23, leap 00, trust 000
refid [129.187.254.32], root delay 0.013397, root dispersion 0.028259
reference time: e1914386.ed1f6f8a Tue, Dec 3 2019 22:26:46.926
originate timestamp: e19145c2.49c7ed54 Tue, Dec 3 2019 22:36:18.288
transmit timestamp: e19145c2.447e4a33 Tue, Dec 3 2019 22:36:18.267
filter delay: 0.06436 0.06421 0.06435 0.06433
---- ---- ---- ----
filter offset: 0.001164 0.001273 0.001315 0.001263
---- ---- ---- ----
delay 0.06421, dispersion 0.00000, offset 0.001273

server 5.9.113.140, port 123
stratum 3, precision -24, leap 00, trust 000
refid [187.182.182.166], root delay 0.013184, root dispersion 0.069717
reference time: e19142cf.992859a5 Tue, Dec 3 2019 22:23:43.598
originate timestamp: e19145c2.7c5eacf3 Tue, Dec 3 2019 22:36:18.485
transmit timestamp: e19145c2.77af13e5 Tue, Dec 3 2019 22:36:18.467
filter delay: 0.06596 0.06598 0.06607 0.06613
---- ---- ---- ----
filter offset: -0.002037 -0.002042 -0.002029 -0.001975
---- ---- ---- ----
delay 0.06596, dispersion 0.00000, offset -0.002037

3 Dec 22:36:18 ntpdate[21274]: adjust time server 194.25.134.196 offset -0.002560 sec

jon · 3 December 2019 20:42

I’ve got timecheck in crontab also. I just don’t remember quite what it does (or why):

github.com

ipfire/ipfire-2.x/blob/master/src/scripts/timecheck

#!/bin/sh
###############################################################################
#                                                                             #
# IPFire.org - A linux based firewall                                         #
# Copyright (C) 2007-2022  IPFire Team  <info@ipfire.org>                     #
#                                                                             #
# This program is free software: you can redistribute it and/or modify        #
# it under the terms of the GNU General Public License as published by        #
# the Free Software Foundation, either version 3 of the License, or           #
# (at your option) any later version.                                         #
#                                                                             #
# This program is distributed in the hope that it will be useful,             #
# but WITHOUT ANY WARRANTY; without even the implied warranty of              #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the               #
# GNU General Public License for more details.                                #
#                                                                             #
# You should have received a copy of the GNU General Public License           #
# along with this program.  If not, see <http://www.gnu.org/licenses/>.       #
#                                                                             #
###############################################################################

This file has been truncated. show original

jon · 3 December 2019 20:45

Try running:

/usr/local/bin/settime $(cat /var/ipfire/time/settime.conf)

same issue?

hjkl · 3 December 2019 20:46

I think I know what /var/ipfire/time/enable does: it appears if I set up the Ipfire as local Time server for local network. ( menu Services > Time Server)
I will stop that settings and rely on external NTP servers delivered by DHCP to all my LAN clients.

This should erase the enable file and therefore timecheck will do…nothing…

jon · 3 December 2019 20:48

just for future reference… I found this page for troubleshooting:

hjkl · 3 December 2019 20:52

Found the problem:

cat /var/ipfire/time/settings
UPDATE_METHOD=manually
VALID=yes
NTP_ADDR_2=80.96.120.253
ENABLECLNTP=off
ENABLENTP=on
UPDATE_VALUE=1
ENABLESETONBOOT=on
UPDATE_PERIOD=daily
NTP_ADDR_1=80.96.196.58

None of the addresses in that file ever work: those are some retires US based NTP servers…

ntpdate -dv 80.96.196.58
3 Dec 22:49:20 ntpdate[22288]: ntpdate 4.2.8p13@1.3847-o Thu May 2 20:06:36 UTC 2019 (1)
Looking for host 80.96.196.58 and service ntp
host found : 80.96.196.58
transmit(80.96.196.58)
transmit(80.96.196.58)
transmit(80.96.196.58)
transmit(80.96.196.58)
80.96.196.58: Server dropped: no data

3 Dec 22:49:29 ntpdate[22288]: no server suitable for synchronization found

So ntp.conf was ok, but /var/ipfire/time/settings had to be cleaned also.

Thanks! All good now!

jon · 3 December 2019 20:55

I do not have my IPFire box setup that way. I found some odd bugs and decided not to try it. I am trying to locate my old notes, from 1 year ago, to find out why. I think it had to do with the flag Provide time to local network (but I am really really guessing!)

jon · 3 December 2019 20:59

I think that should have been reset during the: /etc/rc.d/init.d/ntp restart

jon · 3 December 2019 21:01

Glad you got it working!

hjkl · 3 December 2019 21:07

No, it did not. I did restarted ntp service, but that file remained unchanged.
That file is edited by time.cgi ( menu Services > Time Server)…
Why there are 2 places to define NTP servers (ntp.conf and /var/ipfire/time/settings) is beyond my understanding.

But now I know there are 2 config files to check…

jon · 3 December 2019 21:42

Just so I can correct the post (the original post on the old forum)…
Were the NTP servers on the NTP Configuration time.cgi webpage correct? Or were they the same as the NTP servers in the /var/ipfire/time/settings file?

From what I can figure out: The NTP Configuration webpage @ https://ipfire:444/cgi-bin/time.cgi provides information to /var/ipfire/time/settings and so I don’t touch the settings file.

I think the answer is my instructions must be more clear and the NTP servers on the NTP Configuration webpage @ https://ipfire:444/cgi-bin/time.cgi MUST match the servers added to the /etc/ntp.conf file.

The two entities (ntp.conf and /var/ipfire/time/settings) are not linked since my suggested changes are not (or were not) expected.

jon · 3 December 2019 22:25

This should make things easier/better. Set the NTP servers at the NTP Configuration webpage @ https://ipfire:444/cgi-bin/time.cgi. Then click Save.

Copy the script below to the IPFire box. I named it ntp-enhance.sh. Run the script with:

bash ntp-enhance.sh

Then use:

grep ntp /var/log/messages

to look for unusual NTP messages or errors.

#!/bin/bash
#
#	reconfigure ntp.conf file and add ntp servers
#
set -e

NTPconf="/etc/ntp.conf"

#	get current time setting
settingsFile=$(< /var/ipfire/time/settings)

#	get current ntp servers from settings file
NTP_ADDR_1=$(echo "$settingsFile" | grep NTP_ADDR_1)
NTP_ADDR_1=${NTP_ADDR_1##*=}

NTP_ADDR_2=$(echo "$settingsFile" | grep NTP_ADDR_2)
NTP_ADDR_2=${NTP_ADDR_2##*=}

#	update ntp.conf file using above ntp servers
newNTP=$(cat <<HEREDOC
disable monitor
restrict default nomodify notrap nopeer
restrict 127.0.0.1
server  $NTP_ADDR_1 prefer
server  $NTP_ADDR_2
server  127.127.1.0
fudge   127.127.1.0 stratum 10
driftfile /etc/ntp/drift
HEREDOC
)

#	make a backup of ntp.conf file
cp -p "$NTPconf" "$NTPconf - "$(date +%s)

echo "$newNTP" > "$NTPconf"

#	restart the ntp server
/etc/rc.d/init.d/ntp restart

logger -t ntp "updated ntp.conf file";

anon42188109 · 29 October 2020 16:53

I don’t know if ntp issues are solved but my messages show TIME_ERROR: 0x2041

this is a default core 151 system

[root@ipfire ~]# ntpq -p
localhost.localdomain: timed out, nothing received
***Request timed out

jon · 29 October 2020 17:56

Hi Paul!

Do you run the modified ntp.conf like above?

If so then the ntp.conf configuration may have changed.

do a:

cat /etc/ntp.conf

and post the results.

FYI - I am running CU 151 and ntpq -p works as expected for me. So I am guessing the /etc/ntp.conf file may be the original settings.

anon42188109 · 29 October 2020 18:20

Hi Jon – I did some mods on /etc/ntp.conf, restarted ntpd, and ntpq -p works. Thanks.

disable monitor
restrict default nomodify noquery
restrict 127.0.0.1
server  time1.google.com prefer
server  time2.google.com 
server  127.127.1.0 
fudge   127.127.1.0 stratum 10
driftfile /etc/ntp/drift