RPI with IPfire core 185. Maybe the problem was already in 184.
IPfire is configured with transparent web proxy and it was working OK in the past.
I found I cannot update Ubuntu PC that was connected to RPI with IPfire. When I tried curl archive.canonical.com/ I received error “connection refused”. After some time I have found that squid at RPI is not running`. I tried to start it but it was not started:
[root@rpifire proxy]# /etc/init.d/squid status
/usr/sbin/squid is not running.
/usr/lib/squid/unlinkd is not running.
[root@rpifire proxy]# /etc/init.d/squid start
Creating Squid swap directories... [ OK ]
Starting Squid Proxy Server... [ OK ]
[root@rpifire proxy]# /etc/init.d/squid status
/usr/sbin/squid is not running.
/usr/lib/squid/unlinkd is not running.
In file /var/log/messages I have found that it has some error and exits:
Apr 23 12:39:03 rpifire squid[3624]: Squid Parent: squid-1 process 3627 exited with status 0
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: will start 1 kids
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: (squid-1) process 3639 started
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: squid-1 process 3639 exited with status 1
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: (squid-1) process 3653 started
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: squid-1 process 3653 exited with status 1
Apr 23 12:39:04 rpifire squid[3636]: Squid Parent: (squid-1) process 3670 started
Apr 23 12:39:05 rpifire squid[3636]: Squid Parent: squid-1 process 3670 exited with status 1
Apr 23 12:39:05 rpifire squid[3636]: Squid Parent: (squid-1) process 3678 started
Apr 23 12:39:05 rpifire squid[3636]: Squid Parent: squid-1 process 3678 exited with status 1
Apr 23 12:39:05 rpifire squid[3636]: Squid Parent: (squid-1) process 3684 started
Apr 23 12:39:06 rpifire squid[3636]: Squid Parent: squid-1 process 3684 exited with status 1
Apr 23 12:39:06 rpifire squid[3636]: Squid Parent: squid-1 process 3684 will not be restarted for 3600 seconds due to repeate
d, frequent failures
Apr 23 12:39:06 rpifire squid[3636]: Exiting due to repeated, frequent failures
I tried to run squid in debug mode usr/sbin/squid -d1 and I noticed a problem with permissions on file swap.state:
[root@rpifire proxy]# ll /var/log/cache/
total 72
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 00
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 01
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 02
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 03
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 04
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 05
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 06
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 07
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 08
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 09
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0A
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0B
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0C
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0D
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0E
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0F
-rw-r--r-- 2 root root 4326 Mar 31 03:40 swap.state
-rw-r--r-- 1 squid squid 0 Mar 31 03:37 swap.state.last-clean
I changed ownership of file swap.state:
[root@rpifire proxy]# chown squid:squid /var/log/cache/swap.state
and I was able to start web proxy:
[root@rpifire proxy]# /etc/init.d/squid status
/usr/sbin/squid is not running.
/usr/lib/squid/unlinkd is not running.
[root@rpifire proxy]# /etc/init.d/squid start
Creating Squid swap directories... [ OK ]
Starting Squid Proxy Server... [ OK ]
[root@rpifire proxy]# /etc/init.d/squid status
squid is running with Process ID(s) 5158 5155.
/usr/lib/squid/unlinkd is not running.
I do not know why owner of file /var/log/cache/swap.state was root. I had not touch that file. This is RPI and filesystem is at microSDHC card and shutdown process is not always clean so it is possible that I just captured filesystem in inconsistent state. I am not really sure how this state was created. It could be a bug in squid…
This is state of the directory after fix. It looks like squid was in trouble for some time. This RPI with IPfire is a test system, so I missed that web proxy was down. I tested it deeply today because I upgraded from 184 to 185…
[root@rpifire proxy]# ll /var/log/cache/
total 68
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 00
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 01
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 02
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 03
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 04
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 05
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 06
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 07
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 08
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 09
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0A
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0B
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0C
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0D
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0E
drwxr-xr-x 258 squid squid 4096 Jul 26 2023 0F
-rw-r--r-- 1 squid squid 72 Apr 23 12:50 swap.state
Other interesting point is that file swap.state owned by root is hardlinked with other file, unfortunately I missed that detail before the fix and cannot tell now what was the other file. I had similar issue with ntp a month ago.
-rw-r--r-- 2 root root 4326 Mar 31 03:40 swap.state
If this will happen next time, I will run find / -samefile /var/log/cache/swap.state
Other useful command is find /var -type f -links +2, it finds all files that are hardlinked in /var and sub-directories. I cannot see any such file…