Hi. Normally I use scp and not sftp so I ran a comparison on my systems.
I transferred 48 files totalling approx 380MBytes from IPFire to my desktop. I used scp and then also sftp via Filezilla.
With scp the transfer took 6 seconds and with sftp it took 11 seconds. I checked the logs with Filezilla and there were no dropped connections occurring.
So it looks like there is some other issue on your network. Maybe check out the network hardware is all okay or try some other types of network transfers and see if there are any dropped connections then.
Is there anything in the machine logs of IPFire or the desktop machine running Filezilla that give any clues on what might be happening when the connections are being dropped?