fermitools / ingestd

File ingest daemon

FTSLight3 (Ingestion Daemon)

Making the tarball

$ git clone https://github.com/fermitools/ingestd
$ cd ingestd/FTSLight3
$ make
...
Tar file /tmp/ivm/FTS-light3_3.4.tar is ready

ingestd's Issues

Allow specific data directories on different target servers

Currently if one server has a data directory of /data/DAQ/rawdata
and another one has /data/work/data/rawdata, for instance,
then both of those directories must be scanned on all servers, even if they do not exist, putting
all kinds of annoyance error messages in the logs.

Would be nice to be able to configure
server1
directory1
directory2
server2
directory3
directory4

Change FTS3 options that ingestd uses to be able to configurably configure an FTS3 activity and to add checksum checking to the FTS3 transfer

Recent debugging shows that Ingestd is submitting FTS3 transfers without requesting checking the checksum and with no FTS3 activity specified. We need option to have a activity specified in the ingestd config file which is then added to the FTS3 call,
and also to include the checksum checking in the FTS3 options that are submitted. Same was recently added to DeclaD.