IPFire went down last night, can't find cause

Networking Web Proxy
1

Last night at 10:30 at night I started getting calls that the staff on second and third shift could not get access to the internet and email was not working (externally hosted). Internal LAN traffic was working fine.

I have searched through log files and I have not found anything about a core cause. At 19:50:37 /var/log/messages makes a distinct change in partern and after that point the log is flooded with unbound messages with ‘upstream server timeout’

image
image1903×781 50.9 KB

From home I could ping the work Red interface’s Default Gateway IP (then end of our associated IP block) and the DNS servers we use, but I could not ping the IP address of the red interface.

The connection to websites and e-mail was down until the next morning when I got in, and rebooting IPFire fixed the issue, but of course I want to find out what caused the issue and how I can prevent it from happening in the future.

Where do I look either in the web interface or in the shell to find out what happened? If you had this issue then how would you find the core cause?

Chris

2

This looks like problems of the name server. There are no messages about loss of the red connection. Do you known whether there was a problem with the ISP?
Which DNS servers do you use?

Bernhard

3

I could ping the gateway device of the red interface. On our block we have .56 - .62 (not putting the whole IP here for security reasons) and the red interface had a static assignment of .57 the first usable IP, and .62 was the gateway. I could ping .62 but I could not ping .57.

The fact that I restarted the router in the morning then things came up normally to me indicates that the problem was with IPFire and not the ISP, but it is possible there was a problem with the ISP which which was fixed overnight at some time, but IPFire did not auto recover after the problem was fixed, but restarting in the morning did. I have no way of knowing.

DNS:

image
image902×230 16.7 KB

1 Like
4

If your ISP lost the connection for some reason and it was down for longer than 30 minutes then IPFire’s red interface would have timed out and would not recognise that the line came back after that.

Rebooting or stopping and starting the red dhcpcd interface would restart the check for a connection.

If the red interface did lose its signal then this should show up in the Logs - System Logs under RED

1 Like
5
10:10:33 dhcpcd[7731] : red0: adding default route via XX.XX.XX.62
10:10:33 dhcpcd[7731] : red0: adding route to XX.XX.XX.56/29
10:10:33 dhcpcd[7731] : red0: leased XX.XX.XX.60 for 86400 seconds
10:10:28 dhcpcd[7731] : red0: probing address XX.XX.XX.60/29
10:10:28 dhcpcd[7731] : red0: offered XX.XX.XX.60 from XX.XX.XX.62
10:10:27 dhcpcd[7731] : red0: soliciting a DHCP lease
10:10:26 dhcpcd[7731] : red0: soliciting an IPv6 router
10:10:26 dhcpcd[7731] : ipv6_addaddr1: Permission denied
10:10:26 dhcpcd[7731] : red0: adding address fe80:XX:XX:XX:XX:f8dd
10:10:26 dhcpcd[7731] : red0: IAID 6b:XX:XX:dd
10:10:25 dhcpcd[7731] : DUID XX:XX:XX:XX:XX:XX:XX:dc
10:10:25 dhcpcd[7728] : dhcpcd-9.4.1 starting

Yesterday’s logs only shows a handful of logs in the RED area at 10:10 AM. Yesterday we did experience a power outage around 6:00 AM until 10:00 AM, but then things were back up and running normally until the internet went down again at 7:50 PM, and that’s the issue I’m trying to determine the core cause of

6

Not sure if this is helpful this is yesterday’s Collectd:

|09:51:13|collectd[3272]: |Initialization complete, entering read-loop.|
|09:51:13|collectd[3272]: |cpufreq plugin: Found 4 CPUs|
|09:48:31|collectd[3561]: |rrdtool plugin: Shutting down the queue thread. This may take a while.|
|09:48:31|collectd[3561]: |ping plugin: Shutting down thread.|
|09:48:31|collectd[3561]: |collectd: Stopping 1 read threads.|
|09:48:31|collectd[3561]: |Exiting normally.|
|09:31:05|collectd[3561]: |Initialization complete, entering read-loop.|
|09:31:05|collectd[3561]: |cpufreq plugin: Found 4 CPUs|
|09:28:36|collectd[3684]: |rrdtool plugin: Shutting down the queue thread. This may take a while.|
|09:28:36|collectd[3684]: |ping plugin: Shutting down thread.|
|09:28:36|collectd[3684]: |collectd: Stopping 1 read threads.|
|09:28:36|collectd[3684]: |Exiting normally.|
|09:09:23|collectd[3684]: |Initialization complete, entering read-loop.|
|09:09:23|collectd[3684]: |cpufreq plugin: Found 4 CPUs|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-HOSTILE_DROP/ipt_pack ets-DROP_HOSTILE; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-HOSTILE_DROP/ipt_byte s-DROP_HOSTILE; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-SPOOFED_MARTIAN/ipt_p ackets-DROP_SPOOFED_MARTIAN; value time = 1663149702; last cache update = 166314 9702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-SPOOFED_MARTIAN/ipt_b ytes-DROP_SPOOFED_MARTIAN; value time = 1663149702; last cache update = 16631497 02;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYIN/ipt_packets- DROP_INPUT; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYIN/ipt_bytes-DR OP_INPUT; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYOUT/ipt_packets -DROP_OUTPUT; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYOUT/ipt_bytes-D ROP_OUTPUT; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYFWD/ipt_packets -DROP_FORWARD; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-POLICYFWD/ipt_bytes-D ROP_FORWARD; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-NEWNOTSYN/ipt_packets -DROP_NEWNOTSYN; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-NEWNOTSYN/ipt_bytes-D ROP_NEWNOTSYN; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-PSCAN/ipt_packets-DRO P_PScan; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/iptables-filter-PSCAN/ipt_bytes-DROP_ PScan; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/swap/swap_io-out; value time = 166314 9702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/swap/swap_io-in; value time = 1663149 702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/swap/swap-cached; value time = 166314 9702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/swap/swap-free; value time = 16631497 02; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/swap/swap-used; value time = 16631497 02; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/sensors-coretemp-isa-0000/temperature -temp5; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/sensors-coretemp-isa-0000/temperature -temp4; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/sensors-coretemp-isa-0000/temperature -temp3; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/sensors-coretemp-isa-0000/temperature -temp2; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_errors-tun0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_packets-tun0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_octets-tun0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_errors-blue0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_packets-blue0; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_octets-blue0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_errors-orange0; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_packets-orange0; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_octets-orange0; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_errors-red0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_packets-red0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_octets-red0; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_errors-green0; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_packets-green0; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/interface/if_octets-green0; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/cpufreq/cpufreq-3; value time = 16631 49702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/cpufreq/cpufreq-2; value time = 16631 49702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/cpufreq/cpufreq-1; value time = 16631 49702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/cpufreq/cpufreq-0; value time = 16631 49702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes/fork_rate; value time = 166 3149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_pagefaults; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_cputime; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_stacksize; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_code; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_data; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_rss; value time = 16 63149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-mpd/ps_vm; value time = 166 3149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_pagefaults; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_cputime; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_stacksize; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_rss; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-qemu/ps_vm; value time = 16 63149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_disk_ops; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_disk_octets; val ue time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_pagefaults; valu e time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_count; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_cputime; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_stacksize; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_rss; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-openvpn/ps_vm; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_disk_ops; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_disk_octets; valu e time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_pagefaults; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_cputime; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_stacksize; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_rss; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-charon/ps_vm; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_pagefaults; v alue time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_cputime; valu e time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_stacksize; va lue time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_code; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_data; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_rss; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squidguard/ps_vm; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_disk_ops; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_disk_octets; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_pagefaults; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_cputime; value tim e = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_stacksize; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_rss; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-squid/ps_vm; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_pagefaults; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_cputime; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_stacksize; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_rss; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-nmbd/ps_vm; value time = 16 63149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_pagefaults; value t ime = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_count; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_cputime; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_stacksize; value ti me = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_code; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_data; value time = 1663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_rss; value time = 1 663149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |uc_update: Value too old: name = localhost/processes-smbd/ps_vm; value time = 16 63149702; last cache update = 1663149702;|
|05:01:42|collectd[15015]: |Filter subsystem: Built-in target `write': Dispatching value to the `rrdtool' pl ugin failed with status -1.|
7

/var/log/logwatch/2022-09-14:

root@ipfire logwatch]# cat 2022-09-14
You have old files in your logwatch tmpdir (/var/cache/logwatch):
        logwatch.AjIhVCG1
The directories listed above were most likely created by a
logwatch run that failed to complete successfully.  If so, you
may delete these directories.

*** Error: There is no logfile defined. Do you have a /etc/logwatch/conf/logfiles/secure.conf file ?

 ################### Logwatch 7.6 (01/22/22) ####################
        Processing Initiated: Thu Sep 15 00:05:15 2022
        Date Range Processed: yesterday
                              ( 2022-Sep-14 )
                              Period is day.
        Detail Level of Output: 0
        Type of Output/Format: stdout / text
        Logfiles for Host: ipfire.internal.lan
 ##################################################################

 --------------------- dhcpd Begin ------------------------

 Late or spurious Echo Reply for:
    internal.50: 1 Time(s)
    internal.64: 1 Time(s)
    internal.73: 1 Time(s)
    internal.96: 1 Time(s)
    internal.99: 1 Time(s)
    blue.interface.2: 1 Time(s)

 Unknown Entries:
    5 udp packets in 5 too long - dropped: 1 Time(s)


 ---------------------- dhcpd End -------------------------


 --------------------- Kernel Begin ------------------------


 WARNING:  Out of memory killer killed these executables
    openvpn-authent :  1 Time(s)

 WARNING:  Kernel Errors Present
    Buffer I/O error on dev sda4, log ...:  1 Time(s)
    Buffer I/O error on device sda4,  ...:  30 Time(s)
    EXT4-fs (sda4): previous I/O error to superblock de ...:  1 Time(s)
    EXT4-fs error (device sda4): e ...:  1 Time(s)
    EXT4-fs warning (device sda4): ext4_end_bio:344: I/O error 10 writing to in ...:  7 Time(s)
    blk_update_request: I/O error, dev sda, sector ...:  15 Time(s)
    buffer_io_error: 501 callbacks s ...:  1 Time(s)
    buffer_io_error: 733 callbacks s ...:  1 Time(s)

 ---------------------- Kernel End -------------------------


 --------------------- SSHD Begin ------------------------


 SSHD Killed: 2 Times

 SSHD Started: 3 Times

 Users logging in through sshd:
    root:
       <internal.ip> (me.lan.address): 1 Time

 ---------------------- SSHD End -------------------------


 --------------------- Disk Space Begin ------------------------

 Filesystem      Size  Used Avail Use% Mounted on
 /dev/sda4       116G   16G   95G  14% /
 /dev/sda1       110M   48M   54M  48% /boot
 /dev/sda2        32M  270K   32M   1% /boot/efi


 ---------------------- Disk Space End -------------------------


 --------------------- lm_sensors output Begin ------------------------

 coretemp-isa-0000
 Adapter: ISA adapter
 Core 0:       +46.0 C  (high = +90.0 C, crit = +90.0 C)
 Core 1:       +46.0 C  (high = +90.0 C, crit = +90.0 C)
 Core 2:       +44.0 C  (high = +90.0 C, crit = +90.0 C)
 Core 3:       +44.0 C  (high = +90.0 C, crit = +90.0 C)


 ---------------------- lm_sensors output End -------------------------


 ###################### Logwatch End #########################

[root@ipfire logwatch]#
8

Your logs for the red interface don’t show any problem there.

Your openvpn authentication executable was killed because your system ran out of memory. What does the Memory WUI menu item show for the usage and especially around the time that you had the problem. Either you are running very close to using all the memory up on an ongoing basis, or some program executed and used up all the memory and had a higher priority than the openvpn authentication program.

There are also quite a lot of I/O error which doesn’t look good.
What does the SMART status from the WUI Status - Media menu item say about the disk drive.

Your summary log would make me look at potential hardware problems. That would also tie up with the collectd messages about many values being too old. It is not getting any new data and the cache data is too old My first thought on that was that there was a problem getting the required data for the rrdtool to put into the graphs.

The collectd error messages could also mean that you don’t have any memory data in the graphs for the time that the problem happened.

1 Like
9

There was hardly any memory in use for the time in question, yesterday, there was a spike today between 11 AM and 12 PM but nothing went down at the time:

image
image978×469 27.2 KB

image
image899×866 23.4 KB

image
image1010×888 39.8 KB

10

What do you see in the /var/log/messages file around the 7:50 pm time period that the problem happened.

11

That info is in my first post screenshot. /var/log/messages is a VERY large file: 638 MB, 2,543,435 lines.

Just the log info for Sep 14 for 1900 hours is 30,137 lines.

I’m happy to grep through the file and post output but I don’t precisely know what to look for. I may be looking for a needle in a warehouse full of needles.

12

Hi all,
having here the same log entries from the OOM killer log

Sep 15 19:29:50 ipfire kernel: openvpn-authent invoked oom-killer: gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Sep 15 19:29:50 ipfire kernel: oom_kill_process.cold+0xb/0x10
Sep 15 19:29:50 ipfire kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Sep 15 19:29:50 ipfire kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=openvpn-authent,pid=12228,uid=0
Sep 15 19:29:50 ipfire kernel: Out of memory: Killed process 12228 (openvpn-authent) total-vm:1177692kB, anon-rss:854540kB, file-rss:1252kB, shmem-rss:0kB, UID:0 pgtables:2300kB oom_score_adj:0

Sep 15 19:48:30 ipfire kernel: Out of memory: Killed process 12614 (openvpn-authent) total-vm:1876516kB, anon-rss:1438396kB, file-rss:68kB, shmem-rss:0kB, UID:0 pgtables:3624kB oom_score_adj:0

and the memory graph at this time

memory_graph_oom
memory_graph_oom1446×450 40.3 KB

whereby it seems that ‘openvpn-authent’ process reaches the OOM scores limit and invokes the kernel OOM but this should not causes IPFire to get down since the highest scored processes will be suspended/killed…

Nevertheless the OOM seems a bit strange to me ???

Best,

Erik

1 Like
13

That is a big log file.

Looking at your first screenshot, at 19:50:37 something has happened that causes unbound to be unable to get any responses from any of the DNS servers.

None of the entries immediately above the unbound errors would be expected to stop IPFire from working.
The only suggestion I have is to look at the log entries from 19:50:27 back to something like 19:50:25 or 19:50:22 and see if there are any entries that look unusual (not normal firewall drops or forwards).

If there is nothing within a 2 sec to 5 sec max window then anything earlier is unlikely to be relevant to the stoppage. I would expect problems to show in the log within a quite short period of time to unbound showing the errors.

If there is nothing in the logs when you scan them then we are running out of ideas. If it happens again (hopefully not but just in case) then before restarting IPFire check from the console if RED is connected to the ISP or not.As your RED log showed no problems in it for the involved time then that means either that losing the RED connection was not the problem or something happened so quickly with RED that nothing was able to be logged in time.

Sorry I can’t think of anything further at the moment. Maybe others have further ideas.

1 Like
14

I would definitely agree with you Erik.

I have checked my memory graph and I don’t see any peak like that at all over the whole of the last year. Currently running on CU169. The used memory has varied between 8% and a max of 27%

I did a grep in my messages log for “Out of memory” and nothing showed up for over the last 5 days.

15

Hi Adolf,
have yesterday discovered it by checking some OpenVPN development and debugging. Since i run currently OpenVPN-2.6.dev version i first wasn´t sure where it comes from but since the log entries appears also at @cwensink it seems that it has nothing to do with the OpenVPN dev binary. Have checked (clicked around) the OpenVPN WUI and there it seems somehow appears the OOM by calling /usr/sbin/openvpn-authentication.
Checked via script → 

#!/bin/bash
# Displays running processes in descending order of OOM score
printf 'PID\tOOM Score\tOOM Adj\tCommand\n'
while read -r pid comm; do
	[ -f /proc/$pid/oom_score ] && [ $(cat /proc/$pid/oom_score) != 0 ] && printf '%d\t%d\t\t%d\t%s\n' "$pid" "$(cat /proc/$pid/oom_score)" "$(cat /proc/$pid/oom_score_adj)" "$comm";
done < <(ps -e -o pid= -o comm=) | sort -k 2nr

# EOF

the OOM scores of the processes and there openvpn-authent appears at the very top → 

PID	OOM Score	OOM Adj	Command
5272	981		0	openvpn-authent
6830	810		0	openvpn-authent
2018	772		0	Suricata-Main
3195	676		0	pmacctd
5743	673		0	tor
1619	670		0	unbound
2327	669		0	squid
3187	669		0	pmacctd
11963	668		0	httpd
12839	668		0	collectd
4444	668		0	unbound-dhcp-le
7013	668		0	openvpn-authent
7934	668		0	zsh
19707	667		0	haproxy
3240	667		0	snmpd
4406	667		0	dhcpd
8146	667		0	oom_scores.sh
8147	667		0	oom_scores.sh
11958	666		0	httpd
1570	666		0	vnstatd
1630	666		0	acpid
1662	666		0	dbus-daemon
1679	666		0	hostapd
2325	666		0	squid
2472	666		0	saslauthd
2473	666		0	saslauthd
2484	666		0	cupsd
2538	666		0	ntpd
2654	666		0	httpd
2704	666		0	fcron
2715	666		0	nginx
2716	666		0	nginx
3227	666		0	avahi-daemon
3228	666		0	avahi-daemon
3445	666		0	agetty
3446	666		0	agetty
3447	666		0	agetty
3448	666		0	agetty
3449	666		0	agetty
3450	666		0	agetty
5424	666		0	dma
5432	666		0	dma
5445	666		0	dma
5447	666		0	dma
5458	666		0	dma
5479	666		0	dma
5483	666		0	dma
5485	666		0	dma
587	666		0	lvmetad
6977	666		0	openvpn
7932	666		0	sshd
8148	666		0	sort
8949	666		0	dma
8951	666		0	dma
8952	666		0	dma
8954	666		0	dma
8955	666		0	dma
8957	666		0	dma
8958	666		0	dma
8959	666		0	dma
8960	666		0	dma
8961	666		0	dma
8963	666		0	dma
8964	666		0	dma
8965	666		0	dma
8989	666		0	dma
9013	666		0	dma
9016	666		0	dma
9017	666		0	dma
9022	666		0	dma
9025	666		0	dma
9027	666		0	dma
9028	666		0	dma
9029	666		0	dma
9037	666		0	dma
9038	666		0	dma
9039	666		0	dma
9040	666		0	dma
9048	666		0	dma
9049	666		0	dma
9055	666		0	dma
2640	656		-16	sshd

may a possible memory leak ???

But this seems to be another topic won´t make here an OT…

Best,

Erik

P.S.: Have also Core 169 running…

3 Likes
16

I have a very similar problem, but in my case not only OpenVPN is terminated but also clamAV and the Tor Node, the OpenVPN server restarts automatically, the Virus Scanner and the Tor Node do not.
Why is the memory running so full?

By the way, I noticed that I also have an error with unbound DNS, which I noticed in connection with DDNS, maybe this is related.

17

Everyone,

Well, problems continue here with no known cause. As suggested when the system went down, I tried just restarting the red interface, nothing. I restarted unbound, nothing. I restarted the proxy: nothing. Only restarting the whole router brings things back online.

I had to reset on Friday, Saturday, Sunday, and 4 times today already in the middle of the production day. Nothing appears in /var/log/messages other than a whole lot of DNS unbound connections timing out.

I am seeing these unusual messages:

14:50:46 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06
14:50:46 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0
14:49:37 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06
14:49:37 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0
14:48:28 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06
14:48:28 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0
14:47:19 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06
14:47:19 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0
14:46:10 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06
14:46:10 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0

Our IP Scheme is 10.5.1.X, 10.1.10.X does match the IP scheme of another offsite location, but that’s connected via IPSec point to point tunnel and I’m seeing these in the logs even when the tunnel is down.

Another strange thing that happened. I saw bursts of traffic from one IP address while looking at a tail -f /var/log/messages this morning and I was seeing bursts of messages from one particular IP address, which happened to be the HR Manager’s business iPhone. I had him turn it off after discovering this, then just before lunch he asked if he could turn it back on, he did and then within 30 seconds the network went down again and I had to restart.

So, I factory reset and erased all settings on the iPhone, then updated versions of iOS.

I have not that phone back on the company WiFi yet. I will let it back on after 5 PM to see what the result is.

I also updated to 2.27 Core update 170, but I still had to reset with 170 installed.

I am setting up a complete replacement IPFire router now, on different hardware in case all of this is hardware related, if anyone has any other thoughts, let me know.

18

Here are the logs for about the last 24 hours, filtering for grep for every error, filtering out all unbound messages.

[root@ipfire log]# cat messages | grep "error" | grep -v "unbound"
Sep 18 00:01:48 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:02:50 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:03:52 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:04:54 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:05:55 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:06:57 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file 2.27-x86_64/lists/server-list.db from any available server. There was an error on the way. Please fix it.
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file lists/packages_list.db from any available server. There was an error on the way. Please fix it.
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file lists/core-list.db from any available server. There was an error on the way. Please fix it.
Sep 18 18:14:39 ipfire openvpnserver[3244]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 18 18:14:39 ipfire openvpnserver[3244]: Linux ip addr del failed: external program exited with error status: 2
Sep 18 18:19:06 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:19:29 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:20:08 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:20:30 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:21:10 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:21:32 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:22:12 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:22:34 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:23:14 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:23:36 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:24:16 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:24:38 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:25:19 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:25:40 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:26:20 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:26:42 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:27:22 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:27:44 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:28:23 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:28:46 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:29:10 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:29:48 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:30:12 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:30:50 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:14 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:30 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:51 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:16 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:35 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:53 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:18 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:39 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:56 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:34:19 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:34:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:35:26 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:35:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:36:29 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:36:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:37:30 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:37:51 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:38:33 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:38:54 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:39:34 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:39:57 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:40:36 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:40:42 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:41:00 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:41:44 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:42:04 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:42:45 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:43:05 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:43:35 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:44:09 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:44:38 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:45:11 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:45:40 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:06 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:15 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:42 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:08 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:18 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:44 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:10 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:19 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:46 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:12 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:24 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:48 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:14 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:26 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:50 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:15 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:27 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:52 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:52:17 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:52:54 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:53:19 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:53:56 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:54:38 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:55:39 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:56:40 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:56:41 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:57:42 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:57:44 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:58:44 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:58:45 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:59:46 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:59:47 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:00:48 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:00:49 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:01:50 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:01:51 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:02:52 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:02:53 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:03:54 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:03:56 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:04:55 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:04:57 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:05:57 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:05:59 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:06:59 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:07:02 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:08:01 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:08:02 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:03 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:05 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:32 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:10:34 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:11:36 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:12:38 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:13:40 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:14:43 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:15:44 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:16:46 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:17:48 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:18:50 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:19:51 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:20:54 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:21:55 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:22:57 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:23:59 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:34:42 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:35:44 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:36:46 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:37:48 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:38:49 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:39:51 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:40:54 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:41:56 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:42:57 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:43:59 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:45:01 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:46:03 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:47:05 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:48:07 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:49:09 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:20:23 ipfire openvpnserver[3243]: 193.42.11.190:22003 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:14 ipfire openvpnserver[3243]: 98.165.45.33:60352 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:30 ipfire openvpnserver[3243]: 98.165.45.33:11452 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:47 ipfire openvpnserver[3243]: 98.165.45.33:58592 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:04 ipfire openvpnserver[3243]: 98.165.45.33:19610 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:19 ipfire openvpnserver[3243]: 98.165.45.33:31406 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:36 ipfire openvpnserver[3243]: 98.165.45.33:62774 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:14 ipfire openvpnserver[3243]: 98.165.45.33:18499 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:31 ipfire openvpnserver[3243]: 98.165.45.33:37273 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:47 ipfire openvpnserver[3243]: 98.165.45.33:13885 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:03 ipfire openvpnserver[3243]: 98.165.45.33:11778 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:20 ipfire openvpnserver[3243]: 98.165.45.33:36554 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:36 ipfire openvpnserver[3243]: 98.165.45.33:47129 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:53 ipfire openvpnserver[3243]: 98.165.45.33:17459 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:09 ipfire openvpnserver[3243]: 98.165.45.33:24850 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:26 ipfire openvpnserver[3243]: 98.165.45.33:51007 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:43 ipfire openvpnserver[3243]: 98.165.45.33:62115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:59 ipfire openvpnserver[3243]: 98.165.45.33:25498 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:15 ipfire openvpnserver[3243]: 98.165.45.33:10191 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:31 ipfire openvpnserver[3243]: 98.165.45.33:31685 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:48 ipfire openvpnserver[3243]: 98.165.45.33:16164 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:04 ipfire openvpnserver[3243]: 98.165.45.33:50565 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:21 ipfire openvpnserver[3243]: 98.165.45.33:2286 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:37 ipfire openvpnserver[3243]: 98.165.45.33:24664 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:54 ipfire openvpnserver[3243]: 98.165.45.33:12861 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:56:35 ipfire openvpnserver[3243]: 98.165.45.33:59822 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:56:52 ipfire openvpnserver[3243]: 98.165.45.33:198 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:08 ipfire openvpnserver[3243]: 98.165.45.33:30452 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:24 ipfire openvpnserver[3243]: 98.165.45.33:12279 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:42 ipfire openvpnserver[3243]: 98.165.45.33:47272 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:58 ipfire openvpnserver[3243]: 98.165.45.33:34154 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:15 ipfire openvpnserver[3243]: 98.165.45.33:45420 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:30 ipfire openvpnserver[3243]: 98.165.45.33:59599 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:47 ipfire openvpnserver[3243]: 98.165.45.33:17750 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:03 ipfire openvpnserver[3243]: 98.165.45.33:44496 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:20 ipfire openvpnserver[3243]: 98.165.45.33:45520 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:36 ipfire openvpnserver[3243]: 98.165.45.33:56239 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:52 ipfire openvpnserver[3243]: 98.165.45.33:62981 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:09 ipfire openvpnserver[3243]: 98.165.45.33:47382 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:25 ipfire openvpnserver[3243]: 98.165.45.33:11457 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:43 ipfire openvpnserver[3243]: 98.165.45.33:50502 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:58 ipfire openvpnserver[3243]: 98.165.45.33:3146 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:01:15 ipfire openvpnserver[3243]: 98.165.45.33:28445 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:04:57 ipfire openvpnserver[3243]: 98.165.45.33:60962 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:14 ipfire openvpnserver[3243]: 98.165.45.33:16295 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:30 ipfire openvpnserver[3243]: 98.165.45.33:17740 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:47 ipfire openvpnserver[3243]: 98.165.45.33:19062 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:03 ipfire openvpnserver[3243]: 98.165.45.33:20865 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:19 ipfire openvpnserver[3243]: 98.165.45.33:18150 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:36 ipfire openvpnserver[3243]: 98.165.45.33:33251 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:52 ipfire openvpnserver[3243]: 98.165.45.33:11056 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:09 ipfire openvpnserver[3243]: 98.165.45.33:37646 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:25 ipfire openvpnserver[3243]: 98.165.45.33:42296 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:42 ipfire openvpnserver[3243]: 98.165.45.33:57712 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:58 ipfire openvpnserver[3243]: 98.165.45.33:64464 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:15 ipfire openvpnserver[3243]: 98.165.45.33:17680 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:31 ipfire openvpnserver[3243]: 98.165.45.33:43938 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:47 ipfire openvpnserver[3243]: 98.165.45.33:53936 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:05 ipfire openvpnserver[3243]: 98.165.45.33:55465 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:20 ipfire openvpnserver[3243]: 98.165.45.33:4318 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:37 ipfire openvpnserver[3243]: 98.165.45.33:30816 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:54 ipfire openvpnserver[3243]: 98.165.45.33:41119 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:17:48 ipfire openvpnserver[3243]: 46.105.91.172:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:19:52 ipfire openvpnserver[3243]: 98.165.45.33:26043 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:09 ipfire openvpnserver[3243]: 98.165.45.33:14863 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:25 ipfire openvpnserver[3243]: 98.165.45.33:55887 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:41 ipfire openvpnserver[3243]: 98.165.45.33:37051 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:57 ipfire openvpnserver[3243]: 98.165.45.33:38252 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:14 ipfire openvpnserver[3243]: 98.165.45.33:17330 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:30 ipfire openvpnserver[3243]: 98.165.45.33:50384 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:46 ipfire openvpnserver[3243]: 98.165.45.33:37890 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:03 ipfire openvpnserver[3243]: 98.165.45.33:28761 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:20 ipfire openvpnserver[3243]: 98.165.45.33:30904 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:36 ipfire openvpnserver[3243]: 98.165.45.33:63368 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:52 ipfire openvpnserver[3243]: 98.165.45.33:3840 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:09 ipfire openvpnserver[3243]: 98.165.45.33:20145 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:25 ipfire openvpnserver[3243]: 98.165.45.33:37720 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:42 ipfire openvpnserver[3243]: 98.165.45.33:38368 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:59 ipfire openvpnserver[3243]: 98.165.45.33:5489 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:24:14 ipfire openvpnserver[3243]: 98.165.45.33:1342 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:24:31 ipfire openvpnserver[3243]: 98.165.45.33:28635 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 22:25:58 ipfire openvpnserver[3243]: 146.88.240.4:34169 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 22:36:03 ipfire openvpnserver[3243]: 187.157.9.79:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:08:12 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:09:14 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:10:16 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:11:18 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:12:20 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:13:22 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:14:24 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting

Sep 19 00:47:42 ipfire openvpnserver[3295]: 98.165.45.33:26377 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:47:59 ipfire openvpnserver[3295]: 98.165.45.33:31836 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:15 ipfire openvpnserver[3295]: 98.165.45.33:30567 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:32 ipfire openvpnserver[3295]: 98.165.45.33:47691 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:49 ipfire openvpnserver[3295]: 98.165.45.33:35090 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:49:05 ipfire openvpnserver[3295]: 98.165.45.33:59484 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:49:21 ipfire openvpnserver[3295]: 98.165.45.33:25550 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:19:34 ipfire openvpnserver[3308]: 2.48.75.8:43190 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:38:17 ipfire openvpnserver[3308]: 198.54.114.247:443 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:44:37 ipfire openvpnserver[3308]: 198.54.114.247:443 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 09:02:19 ipfire charon: 06[IKE] received NO_PROPOSAL_CHOSEN notify error
Sep 19 10:23:26 ipfire charon: 13[IKE] received NO_PROPOSAL_CHOSEN notify error
Sep 19 12:33:16 ipfire openvpnserver[3329]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 19 12:33:16 ipfire openvpnserver[3329]: Linux ip addr del failed: external program exited with error status: 2
Sep 19 13:24:26 ipfire openvpnserver[6319]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 19 13:24:26 ipfire openvpnserver[6319]: Linux ip addr del failed: external program exited with error status: 2
Sep 19 13:52:38 ipfire openvpnserver[6354]: 139.162.73.176:41737 SIGUSR1[soft,tls-error] received, client-instance restarting

There are all of these errors relating to openvpnserver, I don’t recognize these IP’s, could this be a DDOS attach on openvpn until the machine runs out of memory and crashes?

Chris

19

Hmmm,

Race condition perhaps.

BR
Joe.

20

How do you fix a race condition?