Borgbackup - No module named 'borg.testsuite'

Hi,
I’ve switch to testing branch (145) for using the new Borg version. In relation with this thread Borgbackup - msgpack dependency.

But I have this message, when I try to use it :frowning:

[root@ipfire borg]# borg --version
Traceback (most recent call last):
File “/usr/lib/python3.8/site-packages/borg/archiver.py”, line 81, in
from .selftest import selftest
File “/usr/lib/python3.8/site-packages/borg/selftest.py”, line 21, in
from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named ‘borg.testsuite’

I have try to reinstall borgbackup but the same issue is present.

Maybe is it normal in IPFire tesing branch ?
Thanks for your help.

I can confirm receiving this exact error on a clean (RAID1) install of v2.25 u147. Reinstallation of borgbackup via pakfire did not help either.

Manually removing line 81 from “/usr/lib/python3.8/site-packages/borg/archiver.py” appears to resolve this issue (borg --version works).

The missing test suite files appear commented out in config/rootfiles/packages/borgbackup [1], but one wonders why they are needed outside development.

[1] https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=config/rootfiles/packages/borgbackup;h=6fbe614296d03ae07f7085a205ee211aad733b7a;hb=HEAD

Please open a bug for this to make sure this info will not get lost.

https://bugzilla.ipfire.org

Hi,
you can take a look here : https://bugzilla.ipfire.org/show_bug.cgi?id=12438

The error seems to be back in Core 148.

Can confirm. Still exists in IPFire 2.25 (x86_64) - Core Update 149

[root@ipfire]# pakfire list | grep -A 3 borg
Name: borgbackup
ProgVersion: 1.1.13
Release: 6

Output with testsuite folder from github like Patrick wrote in Comment 8 @ https://bugzilla.ipfire.org/show_bug.cgi?id=12438#c8

[root@ipfire]# borg
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 22, in <module>
    from .testsuite.crypto import CryptoTestCase
  File "/usr/lib/python3.8/site-packages/borg/testsuite/crypto.py", line 3, in <module>
    from ..crypto.low_level import AES256_CTR_HMAC_SHA256, AES256_OCB, CHACHA20_POLY1305, UNENCRYPTED, \
ImportError: cannot import name 'AES256_CTR_HMAC_SHA256' from 'borg.crypto.low_level' (/usr/lib/python3.8/site-packages/borg/crypto/low_level.cpython-38-x86_64-linux-gnu.so)

IPFire Version output:

[root@ipfire]# cat /etc/system-release
IPFire 2.25 (x86_64) - core149

Greetings
Tom

Hi all,
am running currently Core 150 testing and tested this one too. Borgbackup has been installed before with version 1.1.11 . The first test results in the same problem as above:

$ borg 
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in <module>
    from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named 'borg.testsuite'

after uninstall and reinstall via Pakfire,

# root @ ipfire-prime in ~ [7:44:47] 
$ /usr/local/bin/pakfire remove borgbackup


PAKFIRE INFO: Packages to remove:
PAKFIRE INFO: borgbackup 	 - 620.00 KB
PAKFIRE INFO: Is this okay? [y/N]
y
PAKFIRE REMV: borgbackup: Decrypting...
borgbackup-1.1.13... 100.00% |=============================>|  601.40 KB
PAKFIRE REMV: borgbackup: Removing files and running post-removing scripts...
PAKFIRE REMV: borgbackup: Finished.


# root @ ipfire-prime in ~ [7:44:59] 
$ /usr/local/bin/pakfire install borgbackup
PAKFIRE RESV: borgbackup: Resolving dependencies...
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-llfuse
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-msgpack


PAKFIRE INFO: Packages to install:
PAKFIRE INFO: borgbackup 	 - 620.00 KB

PAKFIRE INFO: Total size: 	 ~ 620.00 KB

PAKFIRE INFO: Is this okay? [y/N]
y

PAKFIRE INST: borgbackup: Decrypting...
PAKFIRE INST: borgbackup: Copying files and running post-installation scripts...
PAKFIRE INST: borgbackup: Finished.

everything works now fine:

$ borg -h
usage: borg [-V] [-h] [--critical] [--error] [--warning] [--info] [--debug] [--debug-topic TOPIC] [-p] [--log-json]
            [--lock-wait SECONDS] [--bypass-lock] [--show-version] [--show-rc] [--umask M] [--remote-path PATH]
            [--remote-ratelimit RATE] [--consider-part-files] [--debug-profile FILE] [--rsh RSH]
            <command> ...

Borg - Deduplicated Backups

optional arguments:
  -V, --version         show version number and exit

Common options:
  -h, --help            show this help message and exit
  --critical            work on log level CRITICAL
  --error               work on log level ERROR
  --warning             work on log level WARNING (default)
  --info, -v, --verbose
                        work on log level INFO
  --debug               enable debug output, work on log level DEBUG
  --debug-topic TOPIC   enable TOPIC debugging (can be specified multiple times). The logger path is
                        borg.debug.<TOPIC> if TOPIC is not fully qualified.
  -p, --progress        show progress information
  --log-json            Output one JSON object per log line instead of formatted text.
  --lock-wait SECONDS   wait at most SECONDS for acquiring a repository/cache lock (default: 1).
  --bypass-lock         Bypass locking mechanism
  --show-version        show/log the borg version
  --show-rc             show/log the return code (rc)
  --umask M             set umask to M (local and remote, default: 0077)
  --remote-path PATH    use PATH as borg executable on the remote (default: "borg")
  --remote-ratelimit RATE
                        set remote network upload rate limit in kiByte/s (default: 0=unlimited)
  --consider-part-files
                        treat part files like normal files (e.g. to list/extract them)
  --debug-profile FILE  Write execution profile in Borg format into FILE. For local use a Python-compatible file can
                        be generated by suffixing FILE with ".pyprof".
  --rsh RSH             Use this command to connect to the 'borg serve' process (default: 'ssh')

required arguments:
  <command>
    mount               mount repository
    serve               start repository server process
    init                initialize empty repository
    check               verify repository
    key                 manage repository key
    change-passphrase   change repository passphrase
    create              create backup
    extract             extract archive contents
    export-tar          create tarball from archive
    diff                find differences in archive contents
    rename              rename archive
    delete              delete archive
    list                list archive or repository contents
    umount              umount repository
    info                show repository or archive information
    break-lock          break repository and cache locks
    prune               prune archives
    upgrade             upgrade repository format
    recreate            Re-create archives
    with-lock           run user command with lock held
    config              get and set configuration values
    debug               debugging command (not intended for normal use)
    benchmark           benchmark command

# root @ ipfire-prime in ~ [7:46:03] 
$ borg -V
borg 1.1.13

# root @ ipfire-prime in ~ [7:46:16] 
$ cd /tmp

# root @ ipfire-prime in /tmp [7:46:34] 
$ mkdir -p bork-test

# root @ ipfire-prime in /tmp [7:46:43] 
$ cd bork-test 

# root @ ipfire-prime in /tmp/bork-test [7:46:47] 
$ borg init --encryption=repokey-blake2 .
Enter new passphrase: 
Enter same passphrase again: 
Do you want your passphrase to be displayed for verification? [yN]: y
Your passphrase (between double-quotes): "nebeleben"
Make sure the passphrase displayed above is exactly what you wanted.

By default repositories initialized with this version will produce security
errors if written to with an older version (up to and including Borg 1.0.8).

If you want to use these older versions, you can disable the check by running:
borg upgrade --disable-tam .

See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details about the security implications.

IMPORTANT: you will need both KEY AND PASSPHRASE to access this repo!
Use "borg key export" to export the key, optionally in printable format.
Write down the passphrase. Store both at safe place(s).

Not sure why that is, possibly it wasn´t respected while the Core update i think it was Core 147 ?

Best,

Erik

Hi Erik,

I just tried it. Didn’t work for me on a fresh installation of Core 150 x64. Can you please try to create a repo because there was also a different error
ImportError: cannot import name 'AES256_CTR_HMAC_SHA256' from 'borg.crypto.low_level'

This seems to be an x64 problem because on x86 everything is working fine. Also the testsuite directory is missing on x64.

Borg version via pakfire:

Greetings
Tom

Possible fix: https://bugzilla.ipfire.org/show_bug.cgi?id=12438#c9

Still doesn’t work for me, unfortunately (now on core 150)

[root@ipfire ~]# /usr/local/bin/pakfire remove borgbackup


PAKFIRE INFO: Packages to remove:
PAKFIRE INFO: borgbackup 	 - 790.00 KB
PAKFIRE INFO: Is this okay? [y/N]
y
PAKFIRE REMV: borgbackup: Decrypting...
PAKFIRE REMV: borgbackup: Removing files and running post-removing scripts...
PAKFIRE REMV: borgbackup: Finished.

[root@ipfire ~]# /usr/local/bin/pakfire install borgbackup
PAKFIRE RESV: borgbackup: Resolving dependencies...
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-llfuse
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-msgpack


PAKFIRE INFO: Packages to install:
PAKFIRE INFO: borgbackup 	 - 790.00 KB

PAKFIRE INFO: Total size: 	 ~ 790.00 KB

PAKFIRE INFO: Is this okay? [y/N]
y

PAKFIRE INST: borgbackup: Decrypting...
PAKFIRE INST: borgbackup: Copying files and running post-installation scripts...
PAKFIRE INST: borgbackup: Finished.

[root@ipfire ~]# borg -h
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 22, in <module>
    from .testsuite.crypto import CryptoTestCase
  File "/usr/lib/python3.8/site-packages/borg/testsuite/crypto.py", line 3, in <module>
    from ..crypto.low_level import AES256_CTR_HMAC_SHA256, AES256_OCB, CHACHA20_POLY1305, UNENCRYPTED, \
ImportError: cannot import name 'AES256_CTR_HMAC_SHA256' from 'borg.crypto.low_level' (/usr/lib/python3.8/site-packages/borg/crypto/low_level.cpython-38-x86_64-linux-gnu.so)

Hi all,
did now a fresh install of IPFire since i had a lot of additional, external Python modules running on the other machine, am currently in Pakfire testing mode of the upcoming Core 151 an borg runs there without problems:

$ mkdir -p /tmp/bork-test && cd /tmp/bork-test && borg init --encryption=repokey-blake2 .
Enter new passphrase: 
Enter same passphrase again: 
Do you want your passphrase to be displayed for verification? [yN]: y
Your passphrase (between double-quotes): "123456"
Make sure the passphrase displayed above is exactly what you wanted.

By default repositories initialized with this version will produce security
errors if written to with an older version (up to and including Borg 1.0.8).

If you want to use these older versions, you can disable the check by running:
borg upgrade --disable-tam .

See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details about the security implications.

IMPORTANT: you will need both KEY AND PASSPHRASE to access this repo!
Use "borg key export" to export the key, optionally in printable format.
Write down the passphrase. Store both at safe place(s).


# root @ ipfire-prime in /tmp/bork-test [7:34:08] 
$ ls -la
total 76
drwxr-xr-x 3 root root  4096 Oct 11 07:34 .
drwxrwxrwt 4 root root  4096 Oct 11 07:33 ..
-rw------- 1 root root   964 Oct 11 07:34 config
drwx------ 3 root root  4096 Oct 11 07:34 data
-rw------- 1 root root    52 Oct 11 07:34 hints.1
-rw------- 1 root root 41258 Oct 11 07:34 index.1
-rw------- 1 root root   190 Oct 11 07:34 integrity.1
-rw------- 1 root root    16 Oct 11 07:34 nonce
-rw------- 1 root root    73 Oct 11 07:33 README

there has been changes pushed with Core 151 --> https://git.ipfire.org/?p=ipfire-2.x.git&a=search&h=refs%2Fheads%2Fnext&st=commit&s=borg , please check them.

Best,

Erik