Some files are big enough that working on them in memory isn't desirable, or even possible. This code demonstrates how to perform efficient data transfer from S3 that is orders of magnitude faster and more efficient than naiively using MemoryStream
while maintaining compatibility with libraries and packages that work with a Stream
interface. Examples for reading ISO, Zip, JPG and Parquet files are includes. A NuGet package is also available for the library, hosted here on GitHub.
For the full explaination, check out the article on Medium.