B2 CLI Finder not Responding on MBP M2 Max about b2_command_line_tool

[ppolewicz]

Try with a really small number of threads like 2 or even 1, see if that helps. 1-thread has a completely different implementation.

[buzaw0nk]

I tried the pypy install and the default threads and it locked up for a several hours and now it is running ok. It's slow and this snapshot is 1.5tb and after 24 hours I'm less than halfway done. Would this work better on a PC? As it sits, B2 is not a viable option if recovery is going to to take days. I'm trialing the service for production use with client data, and I can only imagine a scenario where an office is down and I can't download the backups.

[ppolewicz]

Cheap NAS devices can have notoriously low performance. I've been doing really fast downloads from B2 using the CLI and it was more than fine. The speed depends on what you run it on. For best performance use a fast CPU, really fast network (DigitalOcean will cut you off at a mere 2Gbps) and the newest version of python. Preferably on linux.

EDIT: perhaps the issue might be with the NAS taking a literal approach to TRIM command, preallocationg space for the target by zeroing it?

[buzaw0nk]

I think it may be the external drive connected to the NAS. Looking into the issue. Computer is an M2 MBP, NAS is QNAP TVS-472XT with an i5, network is 10Gb, but MBR has 2.5Gb connection. I will update if I discover something. Making room on a USB 3.2 G2 external SSD now to test again. Thanks for your help.

[ppolewicz]

You might want to test your device with hdparm and you might also want to test b2 itself on a ram drive - just kill the transfer before you run out of memory.

[buzaw0nk]

Tested with Blackmagic Disk Speed Test on the original target (USB3.0 external connected to NAS) and got all of 12Mbs write speeds, lol. I'll look into why later, but I was able to download a 1.5tb file using the download-file-by-name command successfully, although it did seem to slow down after about 15%. Took 4h24m to complete with 75 threads. The internal drives of the NAS speeds are at around 250mbs write and I will attempt the 6.5Tb file later, I need to rearrange some data first. Thanks for your help.

I will update if I am able to download the larger file.

[ppolewicz]

I'd say this is probably too many threads, though if the array is rebuilding or scanning, that might actually be a correct amount I guess?

[buzaw0nk]

Pawel, I would be remiss if I said I knew enough about threads to make an informed decision. I tried to delve into the concept, but just ended up trying different settings to see what would provide the fastest connection. I have too much on my plate atm to give it the focus it deserves. I've cleared some space on the NAS and I'm ready to try the larger snapshot. How many threads do you recommend?

[ppolewicz]

What type of drives do you have and in which raid type?

[buzaw0nk]

WD Red Pro 10Tb, Raid 5 ZFS, QNAP running QutsHero, 1 Gig cable Internet, 10Gb Network with 2.5Gb connection to Mac.

I was toying with the idea of setting up a VM on the NAS and eliminating the MBP from the loop.

[ppolewicz]

I suggest that you use 24 threads, unless you are in Asia or Australia, then 48.