andersfugmann / aws-s3 Goto Github PK

An upper bound on lwt was added in ocaml/opam-repository#24228, which forces consumers of aws-s3-lwt to use lwt.5.6.1 which is not supported in OCaml 5.1.

I believe that the breaking change added in lwt 5.7 was the removal of catch?

cc @raphael-proust

It would be nice to support generating pre-signed URLs for ephemeral GET and PUT operations.

This would be useful for things like minio for local development/testing and the many S3-compatible service providers out there.

For uploads we should support 100 continue, to avoid sending data if the upload fill fail based on the headers.

This is esp useful when handling redirects.

Add a bigstring interface, and change the internals to one use bigstrings to reduce data copying.

This can be important under high concurrency conditions and when the files/data being worked with are large. It may not be feasible to hold the content in a string all at once.

There is currently an upper bound on lwt in the latest 4.8.0 version released to opam: https://github.com/ocaml/opam-repository/blob/master/packages/aws-s3-lwt/aws-s3-lwt.4.8.0/opam#L16

It seems that master should now work fine with the latest lwt (see #35).

It would be nice to release a new version based on master to remove the current restriction on the lwt version.

Amazon allows for chunked uploads, which is useful for streaming.
The sha calculation can be done per chunk in the datastream, which is exactly what is needed for streamed uploads.

Useful for, among other things, checking the size of an object in S3.

The current Lwt support only works with openssl. Conduit, however, also supports ocaml-tls. It would be useful to support both.

core_unix.v0.15.0 is currently not supported on macos because it depends on ocaml_intrinsics.v0.15.0 which currently does not support macos.

It does seem like a fix is underway, but can we please relax the dependency on core_unix (and core i suppose) to at least v0.14.0 while we wait for a fix?

Thank you

Failed connection attempts seems not to be caught an placed in the or_error result.

(((pid 30112) (thread_id 0)) "2019-11-12 10:07:41.525360589Z"
 "unhandled exception in Async scheduler"
 ("unhandled exception"
  ((monitor.ml.Error
    ("connection attempt timeout" 52.217.40.14:443 src/tcp.ml:120:11)
    ("Raised at file \"src/error.ml\" (inlined), line 9, characters 14-30"
     "Called from file \"src/error.ml\" (inlined), line 5, characters 2-50"
     "Called from file \"src/tcp.ml\", line 120, characters 11-76"
     "Called from file \"src/job_queue.ml\", line 170, characters 6-47"))
   ((pid 30112) (thread_id 38)))))

Either extend the error type with a Connection_failed or place the error under Failed of exn