zmanda / amanda Goto Github PK
View Code? Open in Web Editor NEWAmanda Network Backup
Home Page: https://www.zmanda.com/downloads/
License: Other
Amanda Network Backup
Home Page: https://www.zmanda.com/downloads/
License: Other
all non-ASCII characters are in octal representation (\000) in the index file.
amrecover is run in the user locale, it should be able to display characters in user locale
On recovery, the application is also run in the C locale, but can it accept non-ASCII characters? some can and some can't.
That's why the patch can't be committed.
Also, it can be possible to have filename in a different locale, using the root locale to display them can make it harder.
We should add an amrecover setting that enable the translation of an octal representation to a single bytes.
There is a workaround, octal characters must be preceded by 4 backslash:
amrecover> add \303\275
In the dump data on tape or vtape, at the start of each file, there is a header with a couple of plain text lines, the DLE definition in XML form, and a note for human consumption on how to restore the dump without using Amanda. This XML block contains the DLE exclude list. The header is not encrypted, even if the dump is otherwise encrypted. It might be nice to note in the documentation somewhere that the exclude list will be included in the dump data in cleartext even if the dump is encrypted, since this is not obvious and filenames could be considered sensitive data in some situations.
To my knowledge, these are only referenced in amcheck.c
, and are never actually used there. Verify that, and then remove the code (woo!)
When using, for instance, amfetchdump, to restore from a backup into an arbitrary location, warn/bailout if restoring to the selected device will lead to ENOSPC.
For instance, on IRC, a case has been reported of "gzip exited with return code 1". After some quick investigation from the affected user, it appeared that ENOSPC was the cause. The fact that gzip exited with that particular exit code was unhelpful: the real cause was not found.
While each and every cause cannot be analyzed, I think it is at least possible, before starting to restore a backup to an arbitrary location, to warn the user, if he restores interactively, that restoring this backup to this location can potentially read to ENOSPC, given that:
Yes, there are many corner cases, but a first, not-so-precise implementation can imho be easily achieved.
amcrypt-ossl
and amcrypt-ossl-asym
encrypt their data, but they don’t, as far as I can tell, append any sort of MAC, signature, or other integrity protection. Therefore an attacker could modify the ciphertext without being immediately detected, and potentially induce a related change to the plaintext. In particular, to quote Wikipedia, regarding CBC mode, which both these scripts use:
Note that a one-bit change to the ciphertext causes complete corruption of the corresponding block of plaintext, and inverts the corresponding bit in the following block of plaintext, but the rest of the blocks remain intact.
So modifying the ciphertext would not merely destroy the entire plaintext in a way that is necessarily obvious. If the backup were of a filesystem containing large files, an attacker could poke at a bit of ciphertext and have a very good chance of corrupting a restored file without causing any immediately noticeable damage to e.g. the containing tar structure (and tar doesn’t appear to checksum its file data contents). As a general principle, integrity checks over encrypted data are A Good Thing™.
There is a issue with "exclude" in a dumptype using app_amsamba.
Does not matter what you put in exclude this never match.
I fixed this adding the "r" for tar (smbclient internal command) in app_amsamba.
Now i can use in my dumptype:
exclude "\dir\subdir*.pdf"
exclude append "\dir\subdir\avoid_"
exclude append "\dir\subdir\table?of?values_"
? and * are the valid wildcards, ? is a good idea for spaces.
Change /usr/libexec/amanda/application/amsamba
Replace: $comm .= "X";
With: $comm .= "rX";
Add this line: return if $line =~ /^tar_re_search/;
Like this:
return if $line =~ /^tarmode is now /;
return if $line =~ /^tar_re_search/;
if ($line =~ /dumped (\d+) files and directories/) {
I tried paste a diff file but didint work here.
Wagner Caixeta Rodrigues
wagner {dot} caixeta {in} gmail {dot} com
Here I have designed a logo of the "Amanda" pembahruan. I designed it very simple and minimalist, from the logo there is a special symbol of amanda, which has the symbol "a" and the symbol "BackUp". In color I just follow the theme than Amanda, here is the final result of the logo:
I will provide all the original files that can be edited after assessment.
on arch linux for a while now I've been seeing a segfault with amreport
see it happening with all my configs on 2 servers
doing a backtrace on it i get the following
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b20b24 in Perl_pp_formline ()
from /usr/lib/perl5/core_perl/CORE/libperl.so
(gdb) bt
#0 0x00007ffff7b20b24 in Perl_pp_formline ()
from /usr/lib/perl5/core_perl/CORE/libperl.so
#1 0x00007ffff7ae41f6 in Perl_runops_standard ()
from /usr/lib/perl5/core_perl/CORE/libperl.so
#2 0x00007ffff7a7536e in perl_run ()
from /usr/lib/perl5/core_perl/CORE/libperl.so
#3 0x0000000000400e39 in main ()
perl version info is
This is perl 5, version 20, subversion 0 (v5.20.0) built for x86_64-linux-thread-multi
amanda is on 3.3.6
and was compiled with that version of perl also if that matters
let me know if any more info is required
Right now, the dump details are included at the start of a vtape in cleartext, even if the dump itself is encrypted. The dump details include the exclude list, which reveals filenames. I don’t know why the exclude list needs to be stored at all, but if it does, it would be nice to encrypt it.
trying to build 3.3.5 or 3.3.6 there are perl related compile errors
I found this patch:
https://www.mail-archive.com/[email protected]/msg1230089.html
which solves the compile issue for me
and I'm not sure if its related but amreport is crashing with a segfault trying to get
a human readable report
and thats with the above patch
perl version for arch at this time is:
This is perl 5, version 20, subversion 0 (v5.20.0) built for x86_64-linux-thread-multi
let me know if there is any more info is needed
http://wiki.zmanda.com/glib-docs/glib/glib-Lexical-Scanner.html
Looks like a good replacement for the existing code, doesn't it?
rpmlint checks from the opensuse build service are throwing the following warning to port code to getaddrinfo().
[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/sbin/amoldrecover
[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/sbin/amrecover
[ 329s] amanda.x86_64: I: binary-or-shlib-calls-gethostbyname /usr/lib64/amanda/libndmlib-3.3.9.so
[ 329s] The binary calls gethostbyname(). Please port the code to use getaddrinfo().
Summary
Writing to an S3 device from a recent Ubuntu (tried 19.10 and 20.04) does not work, amanda (version 3.5.1, tried amdump, amlabel and 'amcheck -w') just hangs and never returns. Ubuntu 18.04 (also version 3.5.1) works fine. Downgrading libcurl4 on 19.10 and 20.04 to the version of 18.04 seems to fix the problem (although I haven't tested this in production yet, also because I used a bit of force to downgrade, which I don't want to do in production).
How to reproduce
Install 'amanda-server' on an Ubuntu-20.04 instance (I used a vagrant box based on https://github.com/chef/bento/blob/master/packer_templates/ubuntu/ubuntu-20.04-amd64.json). Configure amanda to use an S3 changer device, and use new virtual tapes. Try to label a tape:
backup@vagrant:~$ amlabel test test-01 slot 1
Reading label...
Found an empty tape.
Writing label 'test-01'...
^C
This does not return (and the virtual tape does not get created). The last lines from amlabel.<timestamp>.debug
are:
Wed Jun 03 11:24:48.012207191 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Content-MD5: lYjkchoZihkmrPD9hdJocg==
Wed Jun 03 11:24:48.012210456 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Content-Length: 40
Wed Jun 03 11:24:48.012213590 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out: Date: Wed, 03 Jun 2020 11:24:50 GMT
Wed Jun 03 11:24:48.012216701 2020: pid 14472: thd-0x562400995600: amlabel: Hdr Out:
Workaround
Downgrade libcurl4. I did that as follows, but no doubt there are better ways:
/etc/apt/sources.list
apt-get update
apt-get install libcurl4=7.58.0-2ubuntu3.8
:# apt-get install libcurl4=7.58.0-2ubuntu3.8
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
curl pollinate ubuntu-server
The following packages will be DOWNGRADED:
libcurl4
0 upgraded, 0 newly installed, 1 downgraded, 3 to remove and 1 not upgraded.
...
Now 'amlabel' works just fine:
backup@vagrant:~$ amlabel test test-01 slot 1
Reading label...
Found an empty tape.
Writing label 'test-01'...
Checking label...
Success!
The relevant lines from amlabel.<timestamp>.debug
now read:
Wed Jun 03 12:28:14.777182823 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Content-MD5: lYjkchoZihkmrPD9hdJocg==
Wed Jun 03 12:28:14.777186131 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Content-Length: 40
Wed Jun 03 12:28:14.777189284 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Date: Wed, 03 Jun 2020 12:28:15 GMT
Wed Jun 03 12:28:14.777192351 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out: Expect: 100-continue
Wed Jun 03 12:28:14.777195436 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr Out:
Wed Jun 03 12:28:14.817005276 2020: pid 15279: thd-0x55e019744600: amlabel: Hdr In: HTTP/1.1 100 Continue
after first getting a segfault, I recompiled due to perl update in arch linux i get the following now:
Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at /usr/share/perl5/site_perl/Amanda/Report/human.pm line 1203.
Compilation failed in require at (eval 13) line 1.
BEGIN failed--compilation aborted at (eval 13) line 1.
perl -v returns
This is perl 5, version 22, subversion 0 (v5.22.0) built for x86_64-linux-thread-multi
Using -fsanitize=address I see:
=================================================================
==23452==ERROR: AddressSanitizer: stack-buffer-overflow on address 0xffffd1e1 at pc 0xf7a82dea bp 0xffffd138 sp 0xffffcd10
READ of size 10 at 0xffffd1e1 thread T0
#0 0xf7a82de9 in __interceptor_strlen.part.31 (/usr/lib/libasan.so.5+0x92de9)
#1 0xf7726904 in g_strdup (/usr/lib/libglib-2.0.so.0+0x69904)
#2 0xf7817bb2 in file_lock_write /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock.c:317
#3 0x5655760c in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:115
#4 0x56557fd4 in locking_master /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:206
#5 0x565593bf in test_inter_proc_locking /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:350
#6 0x56559750 in run_one_test /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:66
#7 0x5655a31f in callinfork /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:120
#8 0x5655a31f in testutils_run_tests /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:272
#9 0x56556e19 in main /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:374
#10 0xf7518732 in __libc_start_main (/lib/libc.so.6+0x18732)
#11 0x56556ebf (/home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/.libs/amflock-test+0x1ebf)
Address 0xffffd1e1 is located in stack of thread T0 at offset 33 in frame
#0 0x565574ff in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:104
This frame has 1 object(s):
[32, 33) 'new_val' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/usr/lib/libasan.so.5+0x92de9) in __interceptor_strlen.part.31
Shadow bytes around the buggy address:
0x3ffff9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffff9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x3ffffa30: 00 00 00 00 00 00 00 00 f1 f1 f1 f1[01]f2 f2 f2
0x3ffffa40: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
0x3ffffa50: 01 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa60: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
0x3ffffa70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa80: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==23452==ABORTING
=================================================================
==23461==ERROR: AddressSanitizer: stack-buffer-overflow on address 0xffffd1a1 at pc 0xf7a82dea bp 0xffffd0f8 sp 0xffffccd0
READ of size 10 at 0xffffd1a1 thread T0
#0 0xf7a82de9 in __interceptor_strlen.part.31 (/usr/lib/libasan.so.5+0x92de9)
#1 0xf7726904 in g_strdup (/usr/lib/libglib-2.0.so.0+0x69904)
#2 0xf7817bb2 in file_lock_write /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock.c:317
#3 0x5655760c in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:115
#4 0x56557fd4 in locking_master /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:206
#5 0x5655907a in test_intra_proc_locking /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:315
#6 0x56559750 in run_one_test /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:66
#7 0x5655a31f in callinfork /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:120
#8 0x5655a31f in testutils_run_tests /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/testutils.c:272
#9 0x56556e19 in main /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:374
#10 0xf7518732 in __libc_start_main (/lib/libc.so.6+0x18732)
#11 0x56556ebf (/home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/.libs/amflock-test+0x1ebf)
Address 0xffffd1a1 is located in stack of thread T0 at offset 33 in frame
#0 0x565574ff in inc_counter /home/abuild/rpmbuild/BUILD/amanda-3.5.1/common-src/amflock-test.c:104
This frame has 1 object(s):
[32, 33) 'new_val' <== Memory access at offset 33 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/usr/lib/libasan.so.5+0x92de9) in __interceptor_strlen.part.31
Shadow bytes around the buggy address:
0x3ffff9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffff9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x3ffffa30: f1 f1 f1 f1[01]f2 f2 f2 00 00 00 00 00 00 00 00
0x3ffffa40: 00 00 00 00 f1 f1 f1 f1 01 f2 f2 f2 00 00 00 00
0x3ffffa50: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2
0x3ffffa60: f2 f2 f2 f2 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
0x3ffffa70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x3ffffa80: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==23461==ABORTING
FAIL amflock-test (exit status: 1)
Problem is that file_lock_write
is called with:
char new_val;
which is a character, not terminated by zero. And thus lock->data = g_strdup(data);
then fails because it's calling strlen
, which is expecting a null terminated string.
Amanda 3.3.9’s driver is deciding not to use a new tape even though there is data left to flush. I will attach a level-9 debug output from the taper. I have tried to understand the logic in tape_action and see why this is happening, but I don’t understand it enough to know where the problem is coming from.
driver.20160223003644.txt
see this thread in the ML :: https://www.mail-archive.com/[email protected]/msg50692.html
amrecover does not interpret the parameters correctly.
$ amrestore --config abt -b 2097152 /dev/nst0 jupi smb_revision
seems to work now ... at least it starts searching.
I don't know why I have to tell that ... but it seems I have a mismatch:
tapetype says 32 kbytes:
define tapetype LTO-4 {
comment "Created by amtapetype; compression disabled; 2017-10-31
sgw"
length 698510208 kbytes
filemark 0 kbytes
speed 36696 kps
blocksize 32 kbytes
}
changer def sets "2 mbytes":
define changer robot {
tpchanger "chg-robot:/dev/sg1"
property "tape-device" "0=tape:/dev/nst0"
device-property "BLOCK_SIZE" "2 mbytes"
device-property "READ_BLOCK_SIZE" "2 mbytes"
property "eject-before-unload" "no"
property "use-slots" "1-24"
changerfile "/etc/amanda/abt/chg-robot-dev-sg1"
}
storage def pulls in both:
define storage abt {
tapepool "abt"
tapetype "LTO-4"
tpchanger "robot"
[..]
}
The rpm spec does not appear to include the files in example directory, which I believe are important for amserverconfig
to use?
The deb rules does appear to include this.
docker run -it --rm centos:7
yum install -y epel-release
yum install -y \
ca-certificates mt-st mutt openssh-clients openssh-server gnuplot perl-JSON \
perl-Encode-Locale gettext mailx libcurl aespipe \
perl-Data-Dumper perl-XML-Simple \
https://www.zmanda.com/downloads/community/Amanda/3.5.1/Redhat_Enterprise_7.0/amanda-backup_server-3.5.1-1.rhel7.x86_64.rpm
amserverconfig test2
...
amserverconfig: ERROR: copy dumptypes failed: No such file or directory
I expected to see the same template.d
folder I see on debian in /var/lib/amanda
so I copy them to /etc/amanda
if needed
I'll curl
them from github to get the files I need.
RPMLINT checks from the openSUSE Build Service throw the following warning.
amanda.x86_64: W: missing-call-to-setgroups-before-setuid /usr/sbin/amoldrecover
This executable is calling setuid and setgid without setgroups or initgroups.
There is a high probability this mean it didn't relinquish all groups, and
this would be a potential security issue to be fixed. Seek POS36-C on the web
for details about the problem.
If I have something like this in disklist
MACHINE /dir {
tar
exclude "./subdir/*"
}
MACHINE /dir/subdir tar
and run
$ amdump CONF MACHINE ^/dir$
or
$ amdump CONF MACHINE =/dir
It works as expected the only backup made is /dir
But if I have something like this
MACHINE /dir {
tar
exclude "./subdir/*"
}
MACHINE /dir/subdir /dir {
tar
include "./subdir/*"
}
the commands
$ amdump CONF MACHINE ^/dir$
and
$ amdump CONF MACHINE =/dir
Don't work as expected, both backups (/dir and /dir/subdir) are made
From "man disklist"
diskname
The name of the disk (a label). In most case, you set your diskname
to the diskdevice and you don't set the diskdevice. If you want
multiple entries with the same diskdevice, you must set a different
diskname for each entry. It's the diskname that you use on the
commandline for any Amanda command. Look at the example/disklist
file for example.
So from this, if it's the diskname that is used in the command, it
seems that the reality is amdump is using the diskdevice instead!!
more info from Nathan Stratton Treadway [email protected]
My understanding of the inner workings of this part of Amanda is quite
fuzzy, but assuming I have followed the program flow behind "amdump"
correctly, it looks like the code in question is found in
server-src/diskfile.c:match_disklist() , which contains the following
lines:
if ([...] &&
(match_disk(sargv[i], dp->name) ||
(dp->device && match_disk(sargv[i], dp->device)))) {
libtool: link: gcc -DNDMOS_CONST_NDMJOBLIB_REVISION="amanda-4.0.0alpha.git.d12a7613" -g -O2 -fno-strict-aliasing -o .libs/amndmjob amndmjob_main.o ndmjob_args.o ndmjob_job.o ndmjob_rules.o ndmjob_main_util.o amndma_tape_simulator.o -Wl,--export-dynamic -pthread -pthread ./.libs/libndmjob.so /export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so ../common-src/.libs/libamanda.so -lm -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lnsl -lresolv -lcrypto -lcurl -pthread -Wl,-rpath -Wl,/usr/local/lib/amanda
/export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference to xdr_ndmp4_data_listen_reply' /export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference to
xdr_ndmp3_config_get_auth_attr_reply'
/export/home/orion/fedora/amanda/amanda/build/ndmp-src/.libs/libndmlib.so: undefined reference to `xdr_ndmp3_config_get_butype_info_reply'
......
Restores (amfetchdump, amrecover) that use state-file positioning / direct access recovery and which are from dumps larger than INT_MAX bytes will not be correctly restored by amgtar because the amgtar_restore uses a signed short int type for storing the previous block.
A long int would do fine, or off_t just to be safe...
From time to time I have this error when I run amstatus
amstatus: bad status on taper SHM-WRITE (dumper): 20 at /usr/lib64/perl5/vendor_perl/Amanda/Status.pm line 929, <$fd> line 443248.
amdump is still running and I can see in the logs that is writing in the Tape
amdump.20191110210001
driver: state time 149906.218 free kps: 10000000 space: 1924739276 taper: writing idle-dumpers: 20 qlen tapeq taper0: 2:0 runq: 0 directq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 149906.218 if default: free 10000000
driver: hdisk-state time 149906.218 hdisk 0: free 1924739276 dumpers 0
driver: result time 149906.218 from taper0: PARTDONE worker0-0 00-00728 B00162L6 657 10485760 "[sec 358.948476 bytes 10737418240 kps 29212.437720 orig-kb 179560674]"
log.20191110210001.0
PART taper "ST:XXX06" "POOL:XXX06" B00162L6 657 xxxxx.xxx.xxxx.xx /dev/vda2 20191110210001 8/-1 3 [sec 358.948476 bytes 10737418240 kps 29212.437720 orig-kb 179560674]
Hello Amanda developers,
Thanks for all your hard work on the Amanda backup system. It really works wonder and is relatively straightforward to setup.
Before I point out a possible problem with amsamba, I would like to emphasize that I am not using a typical setup with Amanda and smbclient. For example, I am using Amanda 3.3.3 which seems to be the current Amanda package for Ubuntu 14.04's package manager. Secondly, I am using smbclient 4.3 which is known to be incompatible with Amanda versions less than 3.3.9 (was it?).
However, that doesn't stop me. I've manually merged necessary changes (from studying github diffs) to amsamba to make 3.3.3 work properly with smbclient 4.x.
I have noticed something odd about amsamba's command_estimate() that prepares smbclient commands "archive 0|1;recurse;du". I agree that du should operate on the current working directory (specified by -D), but it doesn't in smbclient 4.3. It operates directly on the share. More specifically, if you have a Windows shared folder //server/share/subdir ... du operates on //server/share and NOT //server/share/subdir even though -D subdir was explicitly passed and your current working directory in smbclient is subdir. I have verified this by manually executing smbclient and trying du.
Furthermore, the man page documenting smbclient shows that "du" is to be passed a filename and does not appear to mention the behavior of the command in the absence of the filename. Smbclient appears to have never documented the use of "du" in the absence of an argument. Thus, I believe that amsamba is relying on undefined behavior in smbclient.
The fix is simple. You can omit the -D option entirely in command_estimate() and instead change the command to run to:
"archive 0|1;recurse;du \"/subdir\""
Even if there is no subdir, "archive 0|1;recurse;du \"/\"" will still be valid.
And all should work correctly afterward.
Does anyone plan to get automated build tests set up via travis on github?
I use that for a small project and assume this would be very helpful. Maybe I can help here.
The amanda.conf
man page says this, for the include
directive:
They must start with "./" and contain no other "/".
However, unwittingly, I put an include
directive in one of my DLEs which specifies a more deeply nested subdirectory (i.e. ./foo/bar
), and it appears to work just fine. Is the documentation incorrect?
We're using the github IRC notifier now, and it notifies of pushes to any branch. This is OK for 3_1
, 3_2
, etc., but since pushes to trunk go to both the trunk
and master
branches on github, we see needlessly duplicated messages. I don't think there's a good reason to push to trunk anymore.
@dwlocks, I think the script that does the pushes is in Buildbot. Can you make this change?
Amanda should have an SSL-based authentication mechanism. SSL is universally supported these days, and the BSD\* are not nearly secure enough for modern applications. The existing SSH protocol is effective, but tricky to set up (as it requires a “manual” login to exchange host keys) and has its own security implications (remote execution of shell commands using an empty passphrase).
When amdump is entered into the console, there is no indication of progress in backup.
Would it be possible to implement a progress bar (such as the one in wget, apt-get, etc.) that can display time remaining and percentage?
The wiki page for chg-multi has a big red banner saying chg-multi
is deprecated. However, the documentation for the diskflat device, which was added very recently, also claims that chg-multi
is the only way to use diskflat. This is confusing. Is chg-multi
deprecated? If not, why does the documentation say it is? If so, why was a new feature added in 3.4.4 that requires its use?
Amanda-3.5.1 on latest Gentoo Linux.
I can load and unload tapes via amtape, I can even label tapes.
But amcheck fails as well as amflush or amdump.
$ amcheck -s abt
Amanda Tape Server Host Check
-----------------------------
NOTE: Holding disk '/mnt/amhold/abt': 169 GB disk space available, using 168 GB
ERROR: amcheck-device terminated with signal 11
Server check took 1.438 seconds
(brought to you by Amanda 3.5.1)
# dmesg
amcheck-device[31798]: segfault at 8 ip 00007f23c67c56e6 sp 00007ffc8ed07488 error 4 in libc-2.27.so[7f23c6727000+1be000]
#dmesg after failing amflush
[11729.998836] taper[32221]: segfault at 8 ip 00007f6af16e46e6 sp 00007fff5120d358 error 4 in libc-2.27.so[7f6af1646000+1be000]
Right now it doesn't, as a result the build fails at compile time.
In config/macro_archive/ac_prog_swig.m4, we have:
AC_DEFUN([AC_PROG_SWIG],[ AC_PATH_PROG([SWIG],[swig]) if test -z "$SWIG" ; then SWIG='echo "Error: SWIG is not installed. You should look at http://www.swig.org" ; false' elif test -n "$1" ; then
Hmmyeah. I just think it should fail outright.
Hi, I complied and installed amanda on my gnu-linux system.
When I try to run the REST-API-Server, I am getting the following error.
$amanda-rest-server start
The REST-API-PORT must be defined in the global amanda.conf (/usr/local/etc/amanda/amanda.conf) and be larger than 1024
I tried looking for the file in /usr/local/etc/amanda/ but amanda.conf does not exist. I tried adding the REST-API-PORT in the amanda-security.conf but it does not work. I even created a file called amanda.conf in the same path, but the rest server did not start.
Can someone please help me out.
If the amanda.conf setup has "tapecycle inf tapes", amflush will
get into an infinite loop in perl/Amanda/Amflush.pm::roll_amdump_logs():
my $days = getconf($CNF_TAPECYCLE) + 2;
for (my $i = $days-1; $i >= 1; $i--) {
my $a = pop @files;
}
The simple patch for this is:
for (my $i = $days-1; @files && $i >= 1; $i--) {
This exists in 3.4.5 and 3.5.1, but not 3.3.4 (the versions I have on hand).
--- amsamba.original 2013-08-19 00:22:26.532825367 -0300
+++ amsamba 2013-08-19 00:23:41.271825238 -0300
@@ -160,6 +160,7 @@
if (defined $self->{'subdir'}) {
$_ =~ s/^\./$self->{'subdir'}/;
}
+ $_ =~ s/([0-7]{3})/chr oct $1/eg;
print INC_FILE "$_\n";
}
@@ -634,7 +635,7 @@
}
$comm .= " tar c";
if ($#{$self->{exclude}} >= 0) {
- $comm .= "X";
+ $comm .= "rX";
}
if ($#{$self->{include}} >= 0) {
$comm .= "I";
@@ -735,13 +736,13 @@
debug("stderr: " . $line);
return if $line =~ /^Domain=/;
return if $line =~ /^tarmode is now/;
+ return if $line =~ /^tar_re_search set/;
if ($line =~ /dumped (\d+) files and directories/) {
$nb_files = $1;
return;
}
-
# message if samba server is configured with 'security = share'
- return if $line =~ $line =~ /Server not using user level security and no password supplied./;
+ return if $line =~ /Server not using user level security and no password supplied./;
if ($line =~ /^Total bytes written: (\d*)/) {
$size = $1;
return;
+ $_ =~ s/([0-7]{3})/chr oct $1/eg;
Fix a problem when try to restore files in SMB mode with non ascii characters (like áàâãäåāăąèééêëēĕėęěìíîïìĩīĭḩóôõöōŏőùúûüũūŭůäàáâãåæçćĉčöòóôõøüùúûßéèêëýñîìíïş).
- $comm .= "X";
+ $comm .= "rX";
+ return if $line =~ /^tar_re_search set/;
Fix a problem when try to exclude files in dumptype, it makes possible to use WILD Cards ? and *, need smbclient compiled with HAVE_REGEX_H.
- return if $line =~ $line =~ /Server not using user level security and no password supplied./;
+ return if $line =~ /Server not using user level security and no password supplied./;
Fix problem like this one:
Unmatched ( in regex; marked by <-- HERE in m/NT_STATUS_OK opening remote file design_grafico_17a_legislatura - deputadosdep_carlosantonio??3CT VISITADOSmosaico.ai ( <-- HERE design_grafico_17a_legislatura - deputadosdep_carlosantonio??3CT VISITADOS)/ at /usr/libexec/amanda/application/amsamba line 743, line 4.]
Slashes need to be escaped to avoid \ to make part of a regex, like \2013 to be understood as octal followed by a "3".
After this fix look the real smbclient line for the error above:
NT_STATUS_OK opening remote file \design_grafico_17a_legislatura - deputados\dep_carlosantonio\2013\CT VISITADOS\mosaico.ai (\design_grafico_17a_legislatura - deputados\dep_carlosantonio\2013\CT VISITADOS)
The only thing built from this directory is amoldrecover, which is required to communicate with a server older than 2.5.1. I think that era has well and truly passed by now, and we can safely say that a client after 3.3 has no capacity to communicate with a server older than 2.5.1.
The code in this directory is old and crusty, and not tested well. I don't think it's even compiled by buildbot, so it may not even compile anymore.
Can we kill it? Pleease‽
Hi,
There's no security reason to do the reverse DNS lookup when the authentication used is SSH, because the security is based on asymmetric keys and not on the DNS resolution at all.
Please check this thread to get more details about issues with reverse DNS lookup.
In this thread, @martineau propose me a patch that is working perfectly in my case.
I think it can be great to include an option to disable the reverse DNS lookup, or maybe disable it when the auth=SSH.
Per the documentation:
Use amrestore to recover client-encrypted or client-custom-compressed tapes.
Given that amrestore
is somewhat painfully low-level, it would be nice if amrecover
supported client-side-encrypted dumps.
When backing up a Windows (an actual Windows server) share //server/share, command_estimate() will fail when smbclient's "du" reports NT_STATUS_ACCESS_DENIED to \System Volume Information and $RECYCLE.BIN\folder which can never be accessed. Currently, it does not appear to be possible to backup entire Windows shares with amsamba.
smbclient 4.x has been known to have a bug where it stops recursing after encountering NT_STATUS_ACCESS_DENIED, but I have manually verified that smbclient 4.3 continues to properly recurse even in the presence of NT_STATUS_ACCESS_DENIED. It reports the total bytes and exits as expected by amsamba. However, amsamba/sendsize seem to treat anything printed to stderr as fatal even though it could get the estimate.
There is no apparent workaround. There is no way to exclude files/folders in recursions performed by smbclient du. There does not appear to be an option to silence such errors in smbclient either.
amsamba needs to discard NT_STATUS_ACCESS_DENIED messages for low level system/file system information like System Volume Information and $RECYCLE.BIN. It might also discard these errors for matching entries in the exclude list/files.
During the build of amanda package (Amanda 3.5) for openSUSE Tumbleweed we bumped into the following error:
[ 81s] I: Program returns random data in a function
[ 81s] E: amanda no-return-in-nonvoid-function Amanda/Config.c:1848, 2261
For the current master, it means missing return value in the "fail:" block for the following functions:
static int val_t_to_sv(val_t *val, SV **results)
https://github.com/zmanda/amanda/blob/master/perl/Amanda/Config.swg#L923
static int val_t_to_print(val_t *val, SV **results)
https://github.com/zmanda/amanda/blob/master/perl/Amanda/Config.swg#L1363
Thanks for fixing it.
Trying to recover a directory "Projektübersicht".
These commands (inside of amrecover) don't work as it seems not to match:
add P*
add "P*"
add "*sicht"
add *
my workaround is to restore the parent directory, it shows "Projekt\374bersicht"
The title says it all. I don't think there is any point in guessing that in the configure flags.
The point is, not only can configure miss (system configuration changes etc), but in the same vein as RDEV_PREFIX, this can be a security hole. I'd rather the sysadmin specify the devices properly.
This option detects situations where a same name is used, either in a function declaration or in a function itself, which shadows an earlier declaration. Examples:
int c; int f(int c) /* warning here */ { ... } int f2(...) { char *tmp; if (condition) { char *tmp; /* warning here */ } }
Unfortunately, there are some warnings triggered by this already, and some of them are not obvious to fix.
With last versions of openssl the warning message
? data encrypt: *** WARNING : deprecated key derivation used.
? data encrypt: Using -iter or -pbkdf2 would be better.
is being produced by amcrypt-ossl.
It can be corrected by adding -pbkdf2 to the amcrypt-ossl calls to openssl:
if [ "$1" = -d ]; then
# decrypt
"${OPENSSL}" enc -pbkdf2 -d "-${CIPHER}" -nopad -salt -pass fd:3 3< "${PASSPHRASE}"
else
# encrypt
pad | "${OPENSSL}" enc -pbkdf2 -e "-${CIPHER}" -nopad -salt -pass fd:3 3< "${PASSPHRASE}"
fi
As mentioned in the amanda-auth manual page:
When accepting an incoming conneciton, the SSH daemon gives Amanda information about the remote system in the $SSH_CONNECTION environment variable. Amanda parses this information to determine the remote address, and then performs a similar check to that done by the BSD authentications: the forward and reverse DNS entries for the remote host must match. As such, while SSH authentication can cryptographically ensure that the remote system is recognized (since it had a recognized secret key), its assurances about the remote host's identity are weaker and depend on the integrity of the DNS.
Clearly, this is not ideal, given that SSH provides us with cryptographically strong authentication. I would like to propose the following alternative:
In addition to the check already described, Amanda will also optionally accept (either in an environment variable or as a parameter to a command-line option) a hostname. Amanda will do all the validation she already does, but will also, in addition, absolutely demand that the hostname used in access control checks is equal to the one thus provided.
Of course, sometimes users can control the command line being executed via SSH. In this case, obviously there is no improvement in security since the user can provide whatever hostname they want in this option. There is no degradation, though, because Amanda should still perform all existing checks.
What this means, though, is that a security-conscious administrator has the option of locking down a specific key to a specific fixed command line in the Amanda user’s authorized_keys
file, and that fixed command line can include the hostname to whom the key belongs. This prevents that host from impersonating any other host, because their key can only ever execute amandad
with its own name passed in.
If this is a good idea, I could try to put together a patch, if that would help, though it might take me a while. I’d also be happy if someone else did it.
Starting with Amanada 3.4.0, it looks like the old Amanda Changer v1 API was removed and replaced with the v2 perl changers.
Does this mean it is not currently possible, or am I missing something?
I currently went back to 3.3.9 to get chg-manual working.
I have issue with too few issues.
I believe from the core of my being that backup is important. I KNOW that tape backup is cheap while effective and I know that Amanda offers a flexibility that is ABSOLUTELY invaluable. I want Amanda to progress into the 21st century tho, and a lot of that requires EG translations to Python over Perl and...smart bois looking at C so we can write either better C or ASM. Obviously, I don't think machine problems are solved by floating away from the machine. We need some bare-metal. I know there is a commercial UI....but what if it was a user UI? I only ask since I forget what the U in UI stands for...
What do people think? What would you want? What is your use case?
EDIT: user story
I have
define changer my_s3 {
...
device_property "TRANSITION-TO-GLACIER" "1"
...
}
and run amadmin amanda-server-archive no-reuse amanda-server-0001
but still 2 days later nothing changes on s3.
What is the command to trigger the transition?
amdump amanda-server-archive
doesn't do the trick.
They are simply wrappers in order to access the ->v.xxx configuration values, except for two of them, which access other fields, which make them even more confusing.
What's more, theirs being usable as lvalues is touted as a feature in the header file: I call that a bad quirk.
And finally, they are too uncomfortably close in writing to the val_t_to_* function.
I say, drop them.
Doing client-side encryption is nice because it means the server can’t see the client’s data. Ideally, in combination with a locked-down command=
parameter in the client’s authorized_keys
file, it should be possible for the client to not really trust the server. However, right now, AFAICT, the only place you can choose whether to enable client-side encryption or not is the dumptype, which is controlled by the server. It would be nice to have an option on the client which would reject all dump requests that don’t specify a particular encryption program. Just requiring encryption alone isn’t sufficient, since the server could just ask the client to encrypt but use /bin/cat as the program.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.