stor-client's Introduction

stor-client

what is stor?

stor is storage HTTP interface for sha256 files (objects)

features

download retry
concurent download (default 4)

cli

read (parse) SHA256 from STDIN and download it to destinationDir

echo EE2BF0BFD365EBF829F8D07B197B7A15F39760CD14C6D3BFDFBAD2B145CB72B8 | stor-client --storage http://stor.domain.tld .

help

usage: stor-cli [<flags>] <downloadDir>

Flags:
      --help           Show context-sensitive help (also try --help-long and --help-man).
  -u, --storage=http://stor.whale.int.avast.com storage url
      --max=4          max download process
      --devnull        download file to /dev/null
  -v, --verbose        more talkativ output
      --timeout=30s    connetion timeout
      --json           log in json format
      --delay=100ms    exponential retry - start delay time
      --attempts=10    count of attempts of retry
      --suffix=""      downloaded file suffix - like '.dat' => SHA.dat
      --upper          name of file will be upper case (not applied to suffix)
      --s3host=S3HOST  host to s3 endpoint with bucket e.g. https://bucket.s3.eu-central-1.amazonaws.com, if is s3url set, first will be use S3, then fallback to stor
      --s3template="{{.FirstShaByte}}/{{.SecondShaByte}}/{{.ThirdShaByte}}/{{.Sha}}" template to S3 path
      --version        Show application version.

Args:
  <downloadDir>  directory for downloaded files

golang client

golang stor-client library

async example

client := storclient.New(storageUrl, storclient.StorClientOpts{})

client.Start()

for _, sha := range shaList {
	client.Download(sha)
}

downloadStatus := client.Wait()

Recommend Projects