Host runs out of memory resulting in mscp being killed about mscp

[upa]

Hi,

Could you please show the output of mscp with -vvv option and the output of ls -l /localcache/** to investigate the issue.

[ayushbindlish]

@upa
This is just before the kill:

 ls -l /localcache/**
-rw-r--r-- 1 root root     0 Dec 11 11:27 /localcache/245ceadc-de14-43dd-856c-cdf2b13c41b4.aux
-rw-r--r-- 1 root root     0 Dec 11 11:27 /localcache/245ceadc-de14-43dd-856c-cdf2b13c41b4.value
-rw-r--r-- 1 root root     0 Dec 11 11:27 /localcache/8efbfc36-40e6-47c2-9b4a-57b95171bcb4.aux
-rw-r--r-- 1 root root     0 Dec 11 11:27 /localcache/8efbfc36-40e6-47c2-9b4a-57b95171bcb4.value
-rw-r--r-- 1 root root 20480 Dec 11 11:27 /localcache/cache.db
-rw-r--r-- 1 root root    33 Dec 28 08:44 /localcache/dovertestbucket.md5sum
-rw-r--r-- 1 root root     0 Dec 11 06:06 /localcache/nvme_stash

/localcache/dovertestbucket:
total 4
drwxr-xr-x 56 root root 4096 Dec 20 11:40 atlassian

mscp_verbose.log.tar.gz

Also, another observation is that if the directory already exists at the destination, it messes up the location.

This is the command I used:
mscp -vvv -i test.pem /localcache/ [email protected]:/tmp/stash |tee mscp_verbose.log

On the second run it created ocalcache dir on remote host:

 ls -lart ./**
-rw-r--r-- 1 ubuntu ubuntu    0 Jan  9 20:17 ./245ceadc-de14-43dd-856c-cdf2b13c41b4.aux
-rw-r--r-- 1 ubuntu ubuntu   33 Jan  9 20:17 ./dovertestbucket.md5sum

./dovertestbucket:
total 12
drwxrwxr-x  3 ubuntu ubuntu 4096 Jan  9 20:17 .
drwxrwxr-x 12 ubuntu ubuntu 4096 Jan  9 20:17 atlassian
drwxrwxr-x  4 ubuntu ubuntu 4096 Jan  9 20:18 ..

./ocalcache:
total 16
drwxrwxr-x 4 ubuntu ubuntu 4096 Jan  9 20:18 ..
-rw-r--r-- 1 ubuntu ubuntu   33 Jan  9 20:18 dovertestbucket.md5sum
-rw-r--r-- 1 ubuntu ubuntu    0 Jan  9 20:18 245ceadc-de14-43dd-856c-cdf2b13c41b4.aux
drwxrwxr-x 3 ubuntu ubuntu 4096 Jan  9 20:18 .
drwxrwxr-x 3 ubuntu ubuntu 4096 Jan  9 20:18 dovertestbucket

[upa]

Hi,

Thank you for the screenshot and logs. I have determined the reason of OOM. I will fix it soon.

On the second run it created ocalcache dir on remote host:

This is an expected behavior. scp also does so.

# Initial setup: There is srcdir and no dstdir.
ryzen1 ~ > mkdir srcdir
ryzen1 ~ > echo asdf > srcdir/asdf
ryzen1 ~ > ll srcdir
total 4.0K
-rw-rw-r-- 1 upa upa 5 Jan 10 21:18 asdf
ryzen1 ~ > ll dstdir
ls: cannot access 'dstdir': No such file or directory

# scp srcdir as dstdir. So, there is dstdir including the file asdf.
ryzen1 ~ > scp -r srcdir localhost:dstdir
asdf                                          100%    5    63.2KB/s   00:00    
ryzen1 ~ > ll dstdir
total 4.0K
-rw-rw-r-- 1 upa upa 5 Jan 10 21:18 asdf

# scp srcdir as dstdir again. As a result, there is srcdir under dstdir.
ryzen1 ~ > scp -r srcdir localhost:dstdir
asdf                                          100%    5    65.0KB/s   00:00    
ryzen1 ~ > ll dstdir/
total 8.0K
-rw-rw-r-- 1 upa upa    5 Jan 10 21:18 asdf
drwxrwxr-x 2 upa upa 4.0K Jan 10 21:18 srcdir

[ayushbindlish]

@upa For the second issue mscp seems to be creating the dir with a wrong name. The expected dir is "localcache" and it ends up creating "ocalcache"

[upa]

Oh, I see. will investigate.

[upa]

The dev branch now includes the fixes for both issues (the truncated remote dir name and OOM). Could you clone and build the dev branch, and test it in your environment? The OOM issue depends on the number of files being copied. I guess the fixed version should be suitable for almost all cases including yours. If it is ok, I will merge it into the main branch and make a new release.

[upa]

Hi @ayushbindlish

I created the new release v0.1.3 including the fixes for this issue, and it is available on homebrew and ubuntu ppa. I believe the issues are resolved in your env, but if not, please make a new issue with detailed information.

Thanks.