That info is in my first post screenshot. /var/log/messages is a VERY large file: 638 MB, 2,543,435 lines.
Just the log info for Sep 14 for 1900 hours is 30,137 lines.
I’m happy to grep through the file and post output but I don’t precisely know what to look for. I may be looking for a needle in a warehouse full of needles.
Hi all,
having here the same log entries from the OOM killer log
Sep 15 19:29:50 ipfire kernel: openvpn-authent invoked oom-killer: gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Sep 15 19:29:50 ipfire kernel: oom_kill_process.cold+0xb/0x10
Sep 15 19:29:50 ipfire kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Sep 15 19:29:50 ipfire kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=openvpn-authent,pid=12228,uid=0
Sep 15 19:29:50 ipfire kernel: Out of memory: Killed process 12228 (openvpn-authent) total-vm:1177692kB, anon-rss:854540kB, file-rss:1252kB, shmem-rss:0kB, UID:0 pgtables:2300kB oom_score_adj:0
Sep 15 19:48:30 ipfire kernel: Out of memory: Killed process 12614 (openvpn-authent) total-vm:1876516kB, anon-rss:1438396kB, file-rss:68kB, shmem-rss:0kB, UID:0 pgtables:3624kB oom_score_adj:0
and the memory graph at this time
whereby it seems that ‘openvpn-authent’ process reaches the OOM scores limit and invokes the kernel OOM but this should not causes IPFire to get down since the highest scored processes will be suspended/killed…
Nevertheless the OOM seems a bit strange to me ???
Best,
Erik
That is a big log file.
Looking at your first screenshot, at 19:50:37 something has happened that causes unbound to be unable to get any responses from any of the DNS servers.
None of the entries immediately above the unbound errors would be expected to stop IPFire from working.
The only suggestion I have is to look at the log entries from 19:50:27 back to something like 19:50:25 or 19:50:22 and see if there are any entries that look unusual (not normal firewall drops or forwards).
If there is nothing within a 2 sec to 5 sec max window then anything earlier is unlikely to be relevant to the stoppage. I would expect problems to show in the log within a quite short period of time to unbound showing the errors.
If there is nothing in the logs when you scan them then we are running out of ideas. If it happens again (hopefully not but just in case) then before restarting IPFire check from the console if RED is connected to the ISP or not.As your RED log showed no problems in it for the involved time then that means either that losing the RED connection was not the problem or something happened so quickly with RED that nothing was able to be logged in time.
Sorry I can’t think of anything further at the moment. Maybe others have further ideas.
I would definitely agree with you Erik.
I have checked my memory graph and I don’t see any peak like that at all over the whole of the last year. Currently running on CU169. The used memory has varied between 8% and a max of 27%
I did a grep in my messages log for “Out of memory” and nothing showed up for over the last 5 days.
Hi Adolf,
have yesterday discovered it by checking some OpenVPN development and debugging. Since i run currently OpenVPN-2.6.dev version i first wasn´t sure where it comes from but since the log entries appears also at @cwensink it seems that it has nothing to do with the OpenVPN dev binary. Have checked (clicked around) the OpenVPN WUI and there it seems somehow appears the OOM by calling /usr/sbin/openvpn-authentication.
Checked via script →
#!/bin/bash
# Displays running processes in descending order of OOM score
printf 'PID\tOOM Score\tOOM Adj\tCommand\n'
while read -r pid comm; do
[ -f /proc/$pid/oom_score ] && [ $(cat /proc/$pid/oom_score) != 0 ] && printf '%d\t%d\t\t%d\t%s\n' "$pid" "$(cat /proc/$pid/oom_score)" "$(cat /proc/$pid/oom_score_adj)" "$comm";
done < <(ps -e -o pid= -o comm=) | sort -k 2nr
# EOF
the OOM scores of the processes and there openvpn-authent appears at the very top →
PID OOM Score OOM Adj Command
5272 981 0 openvpn-authent
6830 810 0 openvpn-authent
2018 772 0 Suricata-Main
3195 676 0 pmacctd
5743 673 0 tor
1619 670 0 unbound
2327 669 0 squid
3187 669 0 pmacctd
11963 668 0 httpd
12839 668 0 collectd
4444 668 0 unbound-dhcp-le
7013 668 0 openvpn-authent
7934 668 0 zsh
19707 667 0 haproxy
3240 667 0 snmpd
4406 667 0 dhcpd
8146 667 0 oom_scores.sh
8147 667 0 oom_scores.sh
11958 666 0 httpd
1570 666 0 vnstatd
1630 666 0 acpid
1662 666 0 dbus-daemon
1679 666 0 hostapd
2325 666 0 squid
2472 666 0 saslauthd
2473 666 0 saslauthd
2484 666 0 cupsd
2538 666 0 ntpd
2654 666 0 httpd
2704 666 0 fcron
2715 666 0 nginx
2716 666 0 nginx
3227 666 0 avahi-daemon
3228 666 0 avahi-daemon
3445 666 0 agetty
3446 666 0 agetty
3447 666 0 agetty
3448 666 0 agetty
3449 666 0 agetty
3450 666 0 agetty
5424 666 0 dma
5432 666 0 dma
5445 666 0 dma
5447 666 0 dma
5458 666 0 dma
5479 666 0 dma
5483 666 0 dma
5485 666 0 dma
587 666 0 lvmetad
6977 666 0 openvpn
7932 666 0 sshd
8148 666 0 sort
8949 666 0 dma
8951 666 0 dma
8952 666 0 dma
8954 666 0 dma
8955 666 0 dma
8957 666 0 dma
8958 666 0 dma
8959 666 0 dma
8960 666 0 dma
8961 666 0 dma
8963 666 0 dma
8964 666 0 dma
8965 666 0 dma
8989 666 0 dma
9013 666 0 dma
9016 666 0 dma
9017 666 0 dma
9022 666 0 dma
9025 666 0 dma
9027 666 0 dma
9028 666 0 dma
9029 666 0 dma
9037 666 0 dma
9038 666 0 dma
9039 666 0 dma
9040 666 0 dma
9048 666 0 dma
9049 666 0 dma
9055 666 0 dma
2640 656 -16 sshd
may a possible memory leak ???
But this seems to be another topic won´t make here an OT…
Best,
Erik
P.S.: Have also Core 169 running…
I have a very similar problem, but in my case not only OpenVPN is terminated but also clamAV and the Tor Node, the OpenVPN server restarts automatically, the Virus Scanner and the Tor Node do not.
Why is the memory running so full?
By the way, I noticed that I also have an error with unbound DNS, which I noticed in connection with DDNS, maybe this is related.
Everyone,
Well, problems continue here with no known cause. As suggested when the system went down, I tried just restarting the red interface, nothing. I restarted unbound, nothing. I restarted the proxy: nothing. Only restarting the whole router brings things back online.
I had to reset on Friday, Saturday, Sunday, and 4 times today already in the middle of the production day. Nothing appears in /var/log/messages other than a whole lot of DNS unbound connections timing out.
I am seeing these unusual messages:
14:50:46 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06 14:50:46 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0 14:49:37 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06 14:49:37 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0 14:48:28 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06 14:48:28 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0 14:47:19 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06 14:47:19 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0 14:46:10 kernel: ll header: 00000000: ff ff ff ff ff ff 9c 8e 99 c1 b4 80 08 06 14:46:10 kernel: IPv4: martian source 10.1.10.1 from 10.1.10.250, on dev green0
Our IP Scheme is 10.5.1.X, 10.1.10.X does match the IP scheme of another offsite location, but that’s connected via IPSec point to point tunnel and I’m seeing these in the logs even when the tunnel is down.
Another strange thing that happened. I saw bursts of traffic from one IP address while looking at a tail -f /var/log/messages this morning and I was seeing bursts of messages from one particular IP address, which happened to be the HR Manager’s business iPhone. I had him turn it off after discovering this, then just before lunch he asked if he could turn it back on, he did and then within 30 seconds the network went down again and I had to restart.
So, I factory reset and erased all settings on the iPhone, then updated versions of iOS.
I have not that phone back on the company WiFi yet. I will let it back on after 5 PM to see what the result is.
I also updated to 2.27 Core update 170, but I still had to reset with 170 installed.
I am setting up a complete replacement IPFire router now, on different hardware in case all of this is hardware related, if anyone has any other thoughts, let me know.
Here are the logs for about the last 24 hours, filtering for grep for every error, filtering out all unbound messages.
[root@ipfire log]# cat messages | grep "error" | grep -v "unbound"
Sep 18 00:01:48 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:02:50 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:03:52 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:04:54 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:05:55 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 00:06:57 ipfire openvpnserver[3244]: 200.95.174.12:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file 2.27-x86_64/lists/server-list.db from any available server. There was an error on the way. Please fix it.
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file lists/packages_list.db from any available server. There was an error on the way. Please fix it.
Sep 18 03:49:10 ipfire pakfire: Giving up: There was no chance to get the file lists/core-list.db from any available server. There was an error on the way. Please fix it.
Sep 18 18:14:39 ipfire openvpnserver[3244]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 18 18:14:39 ipfire openvpnserver[3244]: Linux ip addr del failed: external program exited with error status: 2
Sep 18 18:19:06 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:19:29 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:20:08 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:20:30 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:21:10 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:21:32 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:22:12 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:22:34 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:23:14 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:23:36 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:24:16 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:24:38 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:25:19 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:25:40 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:26:20 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:26:42 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:27:22 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:27:44 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:28:23 ipfire openvpnserver[3243]: 20.42.201.67:30108 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:28:46 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:29:10 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:29:48 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:30:12 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:30:50 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:14 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:30 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:31:51 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:16 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:35 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:32:53 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:18 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:39 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:33:56 ipfire openvpnserver[3243]: 138.59.208.13:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:34:19 ipfire openvpnserver[3243]: 20.42.201.67:30214 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:34:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:35:26 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:35:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:36:29 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:36:48 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:37:30 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:37:51 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:38:33 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:38:54 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:39:34 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:39:57 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:40:36 ipfire openvpnserver[3243]: 20.42.201.67:30025 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:40:42 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:41:00 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:41:44 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:42:04 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:42:45 ipfire openvpnserver[3243]: 20.42.201.67:30234 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:43:05 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:43:35 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:44:09 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:44:38 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:45:11 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:45:40 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:06 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:15 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:46:42 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:08 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:18 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:47:44 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:10 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:19 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:48:46 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:12 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:24 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:49:48 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:14 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:26 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:50:50 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:15 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:27 ipfire openvpnserver[3243]: 205.178.182.243:1194 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:51:52 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:52:17 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:52:54 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:53:19 ipfire openvpnserver[3243]: 20.42.201.67:30102 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:53:56 ipfire openvpnserver[3243]: 20.42.201.67:30110 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:54:38 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:55:39 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:56:40 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:56:41 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:57:42 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:57:44 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:58:44 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:58:45 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:59:46 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 18:59:47 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:00:48 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:00:49 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:01:50 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:01:51 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:02:52 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:02:53 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:03:54 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:03:56 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:04:55 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:04:57 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:05:57 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:05:59 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:06:59 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:07:02 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:08:01 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:08:02 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:03 ipfire openvpnserver[3243]: 20.42.201.67:30233 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:05 ipfire openvpnserver[3243]: 20.42.201.67:30213 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:09:32 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:10:34 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:11:36 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:12:38 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:13:40 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:14:43 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:15:44 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:16:46 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:17:48 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:18:50 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:19:51 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:20:54 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:21:55 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:22:57 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:23:59 ipfire openvpnserver[3243]: 20.42.201.67:30220 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:34:42 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:35:44 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:36:46 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:37:48 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:38:49 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:39:51 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:40:54 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:41:56 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:42:57 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:43:59 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:45:01 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:46:03 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:47:05 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:48:07 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 19:49:09 ipfire openvpnserver[3243]: 20.42.201.67:30115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:20:23 ipfire openvpnserver[3243]: 193.42.11.190:22003 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:14 ipfire openvpnserver[3243]: 98.165.45.33:60352 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:30 ipfire openvpnserver[3243]: 98.165.45.33:11452 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:45:47 ipfire openvpnserver[3243]: 98.165.45.33:58592 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:04 ipfire openvpnserver[3243]: 98.165.45.33:19610 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:19 ipfire openvpnserver[3243]: 98.165.45.33:31406 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:46:36 ipfire openvpnserver[3243]: 98.165.45.33:62774 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:14 ipfire openvpnserver[3243]: 98.165.45.33:18499 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:31 ipfire openvpnserver[3243]: 98.165.45.33:37273 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:51:47 ipfire openvpnserver[3243]: 98.165.45.33:13885 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:03 ipfire openvpnserver[3243]: 98.165.45.33:11778 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:20 ipfire openvpnserver[3243]: 98.165.45.33:36554 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:36 ipfire openvpnserver[3243]: 98.165.45.33:47129 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:52:53 ipfire openvpnserver[3243]: 98.165.45.33:17459 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:09 ipfire openvpnserver[3243]: 98.165.45.33:24850 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:26 ipfire openvpnserver[3243]: 98.165.45.33:51007 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:43 ipfire openvpnserver[3243]: 98.165.45.33:62115 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:53:59 ipfire openvpnserver[3243]: 98.165.45.33:25498 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:15 ipfire openvpnserver[3243]: 98.165.45.33:10191 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:31 ipfire openvpnserver[3243]: 98.165.45.33:31685 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:54:48 ipfire openvpnserver[3243]: 98.165.45.33:16164 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:04 ipfire openvpnserver[3243]: 98.165.45.33:50565 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:21 ipfire openvpnserver[3243]: 98.165.45.33:2286 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:37 ipfire openvpnserver[3243]: 98.165.45.33:24664 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:55:54 ipfire openvpnserver[3243]: 98.165.45.33:12861 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:56:35 ipfire openvpnserver[3243]: 98.165.45.33:59822 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:56:52 ipfire openvpnserver[3243]: 98.165.45.33:198 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:08 ipfire openvpnserver[3243]: 98.165.45.33:30452 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:24 ipfire openvpnserver[3243]: 98.165.45.33:12279 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:42 ipfire openvpnserver[3243]: 98.165.45.33:47272 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:57:58 ipfire openvpnserver[3243]: 98.165.45.33:34154 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:15 ipfire openvpnserver[3243]: 98.165.45.33:45420 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:30 ipfire openvpnserver[3243]: 98.165.45.33:59599 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:58:47 ipfire openvpnserver[3243]: 98.165.45.33:17750 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:03 ipfire openvpnserver[3243]: 98.165.45.33:44496 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:20 ipfire openvpnserver[3243]: 98.165.45.33:45520 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:36 ipfire openvpnserver[3243]: 98.165.45.33:56239 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 20:59:52 ipfire openvpnserver[3243]: 98.165.45.33:62981 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:09 ipfire openvpnserver[3243]: 98.165.45.33:47382 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:25 ipfire openvpnserver[3243]: 98.165.45.33:11457 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:43 ipfire openvpnserver[3243]: 98.165.45.33:50502 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:00:58 ipfire openvpnserver[3243]: 98.165.45.33:3146 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:01:15 ipfire openvpnserver[3243]: 98.165.45.33:28445 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:04:57 ipfire openvpnserver[3243]: 98.165.45.33:60962 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:14 ipfire openvpnserver[3243]: 98.165.45.33:16295 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:30 ipfire openvpnserver[3243]: 98.165.45.33:17740 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:05:47 ipfire openvpnserver[3243]: 98.165.45.33:19062 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:03 ipfire openvpnserver[3243]: 98.165.45.33:20865 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:19 ipfire openvpnserver[3243]: 98.165.45.33:18150 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:36 ipfire openvpnserver[3243]: 98.165.45.33:33251 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:06:52 ipfire openvpnserver[3243]: 98.165.45.33:11056 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:09 ipfire openvpnserver[3243]: 98.165.45.33:37646 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:25 ipfire openvpnserver[3243]: 98.165.45.33:42296 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:42 ipfire openvpnserver[3243]: 98.165.45.33:57712 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:07:58 ipfire openvpnserver[3243]: 98.165.45.33:64464 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:15 ipfire openvpnserver[3243]: 98.165.45.33:17680 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:31 ipfire openvpnserver[3243]: 98.165.45.33:43938 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:08:47 ipfire openvpnserver[3243]: 98.165.45.33:53936 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:05 ipfire openvpnserver[3243]: 98.165.45.33:55465 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:20 ipfire openvpnserver[3243]: 98.165.45.33:4318 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:37 ipfire openvpnserver[3243]: 98.165.45.33:30816 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:09:54 ipfire openvpnserver[3243]: 98.165.45.33:41119 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:17:48 ipfire openvpnserver[3243]: 46.105.91.172:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:19:52 ipfire openvpnserver[3243]: 98.165.45.33:26043 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:09 ipfire openvpnserver[3243]: 98.165.45.33:14863 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:25 ipfire openvpnserver[3243]: 98.165.45.33:55887 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:41 ipfire openvpnserver[3243]: 98.165.45.33:37051 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:20:57 ipfire openvpnserver[3243]: 98.165.45.33:38252 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:14 ipfire openvpnserver[3243]: 98.165.45.33:17330 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:30 ipfire openvpnserver[3243]: 98.165.45.33:50384 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:21:46 ipfire openvpnserver[3243]: 98.165.45.33:37890 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:03 ipfire openvpnserver[3243]: 98.165.45.33:28761 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:20 ipfire openvpnserver[3243]: 98.165.45.33:30904 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:36 ipfire openvpnserver[3243]: 98.165.45.33:63368 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:22:52 ipfire openvpnserver[3243]: 98.165.45.33:3840 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:09 ipfire openvpnserver[3243]: 98.165.45.33:20145 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:25 ipfire openvpnserver[3243]: 98.165.45.33:37720 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:42 ipfire openvpnserver[3243]: 98.165.45.33:38368 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:23:59 ipfire openvpnserver[3243]: 98.165.45.33:5489 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:24:14 ipfire openvpnserver[3243]: 98.165.45.33:1342 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 21:24:31 ipfire openvpnserver[3243]: 98.165.45.33:28635 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 22:25:58 ipfire openvpnserver[3243]: 146.88.240.4:34169 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 22:36:03 ipfire openvpnserver[3243]: 187.157.9.79:80 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:08:12 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:09:14 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:10:16 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:11:18 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:12:20 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:13:22 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 18 23:14:24 ipfire openvpnserver[3243]: 20.42.201.67:30136 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:47:42 ipfire openvpnserver[3295]: 98.165.45.33:26377 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:47:59 ipfire openvpnserver[3295]: 98.165.45.33:31836 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:15 ipfire openvpnserver[3295]: 98.165.45.33:30567 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:32 ipfire openvpnserver[3295]: 98.165.45.33:47691 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:48:49 ipfire openvpnserver[3295]: 98.165.45.33:35090 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:49:05 ipfire openvpnserver[3295]: 98.165.45.33:59484 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 00:49:21 ipfire openvpnserver[3295]: 98.165.45.33:25550 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:19:34 ipfire openvpnserver[3308]: 2.48.75.8:43190 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:38:17 ipfire openvpnserver[3308]: 198.54.114.247:443 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 08:44:37 ipfire openvpnserver[3308]: 198.54.114.247:443 SIGUSR1[soft,tls-error] received, client-instance restarting
Sep 19 09:02:19 ipfire charon: 06[IKE] received NO_PROPOSAL_CHOSEN notify error
Sep 19 10:23:26 ipfire charon: 13[IKE] received NO_PROPOSAL_CHOSEN notify error
Sep 19 12:33:16 ipfire openvpnserver[3329]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 19 12:33:16 ipfire openvpnserver[3329]: Linux ip addr del failed: external program exited with error status: 2
Sep 19 13:24:26 ipfire openvpnserver[6319]: ERROR: Linux route delete command failed: external program exited with error status: 2
Sep 19 13:24:26 ipfire openvpnserver[6319]: Linux ip addr del failed: external program exited with error status: 2
Sep 19 13:52:38 ipfire openvpnserver[6354]: 139.162.73.176:41737 SIGUSR1[soft,tls-error] received, client-instance restarting
There are all of these errors relating to openvpnserver, I don’t recognize these IP’s, could this be a DDOS attach on openvpn until the machine runs out of memory and crashes?
Chris
Hmmm,
Race condition perhaps.
BR
Joe.
How do you fix a race condition?
Hi all,
@cwensink i would check your OpenVPN logs not only by grepping for errors, may you can find there more informations. Nevertheless, there seems to be something really wrong with your OpenVPN configuration possibly you can also find this messages in the OpenVPN logs " TLS Error: TLS key negotiation failed to occur within 60 seconds", since this message appears very frequent, OpenVPN have decided to create an own page/checklist for this problem → https://openvpn.net/faq/tls-error-tls-key-negotiation-failed-to-occur-within-60-seconds-check-your-network-connectivity/ may you know it and or checked it ?
Another thing, it might be great if you can set the code tags if you post logs (easier to read).
Some ideas to one problem. May it might be also helpful if you sort the different problems (OOM, OpenVPN, Unbound, Smart status) a little more for your investigation but also for the community may some help comes faster ?
Best,
Erik
Not sure if it helps.
I have had no problem with core 169
Core update did add 2factor authentication to OPENvpn.
A feature I’m not using.
Not a tor addon user either as a side note.
How long has this appliance been in service and were any updates or changes made, no matter how minor, before the first crash instance?
What version and core update are you running?
The most worrying thing I’ve seen in this entire post are the drive IO errors. If the drive is failing, it will cause all sorts of random errors. Same can be said for memory though. I get SMART says the drive is healthy, but it’s not entirely reliable. Could be a physical issue as well like a loose power or data cable, maybe a failing port.
Not sure how long you can take this appliance out of service, but if no software changes were made, I would install a new drive, restore the latest backup file and run memtest.
Tuesday night I re-built a new IPFire machine, and the next two days have been quiet, errors for openvpn ended on 9/20 per the messages log. It’s entirely possible that all of this had nothing to do with IPFire and was related to hardware failure or possibly a security breach on the router, but the router is offline now.
Any suggestions on what utilities to run to test the hardware of the device? The machine is a Quad Core Atom processor 8, GB ram, 120 GB ssd Micro ITX Supermicro box 2.7.Ghz, and I am ok with any testing app in an offline environment.
What apps do you guys like to use?
DisturbedDragon,
This unit has been in service for about 5 years, and has had a number of changes in the configuration along the way. It first started having these major problems when running 169, and for troubleshooting I tried updating to 170, but that did not fix the issue. On this appliance there’s a power plug with an external power supply that’s similar to a laptop except the plug screws in on the end (like a water hose).
Chris
With the new machine in place and working I suppose the issue is resolved.
The issue appears to be an acute one. Working fine one day and then became crashy. I’m going with a hardware error again printing out the IO issues logged. You could still troubleshoot the old machine with the steps provided previously since it is now offline.
I will say that upgrading a machine experiencing issues is a bad idea. Updating the software could introduce a new bug compounding existing problems with new ones. Better to resolve any issues then upgrading.
Update to this issue. I now have had 3 different IPFire appliances gone down on me in the last 4 weeks. One is a custom build machine with a Xeon E3 CPU, 4 GB of ram and a WD Black 500 GB sata disk in it, which functioned just fine up until I took it offline for a hardware upgrade. I’ve since tried two different Supermicro Mini ITX Systems, Model 101S-6 P/N SYS-E200-9B, both have 8 GB of ram, and 120 GB SSD’s in them. Both of those units have experienced issues going down and randomly rebooting, with no clear cause. I had one reboot on my on Friday, and 30 minutes ago the system restarted again. Thankfully the system came back online so fast that nobody hardly noticed the downtime. I confirmed by connecting in via bash, and I see the uptime is just 1 minute.
I’m starting to feel like I am cursed. I’m searching /var/log/messages for any indication about what is happening but I am coming up with nothing.
Any suggestions anyone?
Chris,
Please read through the above posts by others. yes again.
Keep in mind you are our eyes in this situation.
There are suggestions above and little response from you (like post #21)
Are you using CU 170 on all three IPFire devices?
Pick one device you want to fix. Look for the time of failure and make note of it (for us and for you!)
Look through the logs starting at that time (time of issue) and go backwards about 10 minutes. If you don’t see anything notable, post the message log for entire 10 min time frame. (make sure we know what time it failed).
Same for the OpenVPN log.
Look through the graphs (like Memory). Look at the same time. Did it spike again?
Also - is this device still running OK? Or did it have the same sudden reboot issue?
I am curious - Have you tried installing a new hard drive?