the uploaded image is damaged about lua-resty-upload

[agentzh]

@jxskiss Will you try adding the b mode to the io.open call? That is,

file = io.open(file_name, 'w+b')

See https://www.lua.org/manual/5.1/manual.html#pdf-io.open

[bungle]

Good catch @agentzh,

I have fixed my lua-resty-reqargs:
https://github.com/bungle/lua-resty-reqargs

And provided a patch for @antonheryanto's lua-resty-post:
antonheryanto/lua-resty-post#3

[agentzh]

Please provide a minimal and standalone example that can easily reproduce the problem on our side. My hunch is that it might be a problem in your own Lua code. Be careful about the streaming API of this library.

[wilsonp]

thanks, agentzh,I upload a zip file including my lua code and the original picture and the uploaded picture,thx again
upload_image_damaged.zip

[bungle]

@wilsonp,

This code does exactly that you are trying:
https://github.com/bungle/lua-resty-reqargs/blob/master/lib/reqargs/reqargs.lua

Could you try that as well?

[wilsonp]

@bungle thanks,I will try it later

[wilsonp]

@bungle Hi,bungle,I run your code,but the result is the same as my code,so I must continue to find the reason,thanks again

[bungle]

@agentzh, it seems like lua-resty-upload or lua-nginx-module corrupts the uploads on Windows (server side).

See also: bungle/lua-resty-reqargs#2

[agentzh]

@bungle This is strange. Are the users using the official Win32 build of OpenResty downloaded from the official site?

[bungle]

I don't know. I try to test this, but I do not have time to do it in next two weeks.

[agentzh]

@bungle Thanks!

[jxskiss]

@agentzh 我也遇到了这个问题，从openresty.org 下载的1.9.15.1-win32版本。
另外，我上传文件的时候使用resty.md5做了md5校验，md5检验和源文件一样，但是保存下来的文件无法打开，保存后的文件md5校验也不对。
使用的代码就是resty-upload模块的示例代码。

在另外，LomoX-Offical/nginx-openresty-windows下载的 1.9.15.1002版本，连上传过程中的md5校验都不对，不知道是什么问题。

[jxskiss]

测试了以下，官网上 ngx_openresty-1.9.3.2-win32 以后的版本都不行，之前的版本没有编译，没测试。

[agentzh]

@jxskiss Please, do not use Chinese here. This place is considered English only. If you
really want to use Chinese, please join and post to the openresty (Chinese)
mailing list instead. Please see https://openresty.org/en/community.html Thanks for
your cooperation.

[agentzh]

@jxskiss Regarding your problem, please provide a minimal and standalone example that I can run and reproduce your issue directly on my side. Again, please do not use Chinese here.

[jxskiss]

Sorry for submit Chinese messages.

Following is the code I use:

    location /upload {
        content_by_lua_block {
            local resty_md5 = require "resty.md5"
            local upload = require "resty.upload"

            local chunk_size = 4096
            local form = upload:new(chunk_size)
            local md5 = resty_md5:new()
            local file

            local upload_path = 'C:/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads/'


            while true do
                local typ, res, err = form:read()

                if not typ then
                    ngx.say("failed to read: ", err)
                    return
                end

                if typ == "header" then
                    local file_name = upload_path .. ngx.time()
                    if file_name then
                        file = io.open(file_name, 'w+')
                        if not file then
                            ngx.say("failed to open file ", file_name)
                            return
                        end
                    end
                elseif typ == "body" then
                    if file then
                        file:write(res)
                        md5:update(res)
                    end
                elseif typ == "part_end" then
                    file:close()
                    file = nil
                elseif typ == "eof" then
                    break
                else
                    -- nothing
                end
            end

            local digest = md5:final()
            md5:reset()

            local str = require "resty.string"
            ngx.say("md5: ", str.to_hex(digest))
        }
    }

---

And the test result:

wsh@lap MSYS /c/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads
$ rm -rf ./*; ls

wsh@lap MSYS /c/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads
$ md5sum ../README.txt
*b3b8205a892b236aad6ffa52e89b3efb ../README.txt

wsh@lap MSYS /c/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads
$ curl -F "filename=@../README.txt" http://127.0.0.1/upload
md5: b3b8205a892b236aad6ffa52e89b3efb

wsh@lap MSYS /c/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads
$ ls
1466237683

wsh@lap MSYS /c/Users/wsh/Desktop/openresty-1.9.15.1-win32/uploads
$ md5sum 1466237683
*82988b3ca2003da5da17cea9dfc857ea 1466237683

---

OS version if needed: Windows 7 SP1 x64

[agentzh]

@jxskiss Thanks for the details. I'll give it a shot once on my own Win32 box when I have a chance.

[jxskiss]

Is there any news for this issue?

[agentzh]

@jxskiss I haven't got a chance to try it out on Windows yet. Sorry.

[jxskiss]

@agentzh Hello again, I found something new about this issue.
When I compared the uploaded README.txt file with the original README.txt file with Vim, I found a new line character was added to every line's end, hope to be helpful.

First lines of the original file content printed by python:
"Name\n README-win32 - README for the Windows 32-bit build of OpenResty\n\nDescription\n The binary distributions of OpenResty can be downloaded from the\n following web page:\n\n https://openresty.org/#Download\n\n To start the NGINX server of the nginx server of the Win32 binary\n distribution of OpenResty:\n\n start nginx\n\n You can also specify the "-p PATH/" option to override the default\n server prefix, as in\n\n cd /path/to/my/openresty/app/\n start nginx -p $PWD\n\n Then you can use the "tasklist" command to check the nginx processes\n running in the background:\n\n C:> tasklist /fi "imagename eq nginx.exe"\n \n Image Name PID Session Name Session# Mem Usage\n ========================= ======== ================ =========== ============\n"

The following is the first lines of the upload file:
"Name\n\n README-win32 - README for the Windows 32-bit build of OpenResty\n\n\n\nDescription\n\n The binary distributions of OpenResty can be downloaded from the\n\n following web page:\n\n\n\n https://openresty.org/#Download\n\n\n\n To start the NGINX server of the nginx server of the Win32 binary\n\n distribution of OpenResty:\n\n\n\n start nginx\n\n\n\n You can also specify the "-p PATH/" option to override the default\n\n server prefix, as in\n\n\n\n cd /path/to/my/openresty/app/\n\n start nginx -p $PWD\n\n\n\n Then you can use the "tasklist" command to check the nginx processes\n\n running in the background:\n\n\n\n C:> tasklist /fi "imagename eq nginx.exe"\n\n \n\n Image Name PID Session Name Session# Mem Usage\n\n ========================= ======== ================ =========== ============\n\n"

[jxskiss]

@agentzh Yes, it works by opening in binary mode, thanks.

wsh@lap MSYS /mnt/d/web/openresty-1.9.15.1-win32/uploads
$ ls
image.jpg README.txt

wsh@lap MSYS /mnt/d/web/openresty-1.9.15.1-win32/uploads
$ curl -F "[email protected]" http://127.0.0.1:88/upload
md5: b3b8205a892b236aad6ffa52e89b3efb

wsh@lap MSYS /mnt/d/web/openresty-1.9.15.1-win32/uploads
$ curl -F "[email protected]" http://127.0.0.1:88/upload
md5: 0f341a98aefe774624a69e4bd46e3f0b

wsh@lap MSYS /mnt/d/web/openresty-1.9.15.1-win32/uploads
$ md5sum *
b3b8205a892b236aad6ffa52e89b3efb *1471609835
0f341a98aefe774624a69e4bd46e3f0b *1471609856
0f341a98aefe774624a69e4bd46e3f0b *image.jpg
b3b8205a892b236aad6ffa52e89b3efb *README.txt

[agentzh]

@jxskiss Awesome. I remembered opening files in the default text mode on Windows can result in fancy Windows CR LF new lines being added automatically.

@wilsonp Will you try the same trick on your side? Opening a file for image data in the default text mode will surely result in corrupted images :)

[agentzh]

@bungle Awesome. Thanks!

[agentzh]

Consider it resolved.