Error while retrieving a file from AWS S3 with Fog about carrierwave-meta

[gzigzigzeo]

This method is called because File instance of the retreived file in some cases contains wrong #content_type property value, so it had to be reassigned with #set_content_type call. Caching of content-type could be done only if uploader is attached to a model. #set_content_type uses the Mime::Type gem. This gem locates content type by file's extension. #original_filename is used to get original file's extension. I think S3 have other way to identify file's content-type even #original_filename is missing or blank. But, in general, it may be CarrierWave's fail.

Could you provide the failing spec?

[bndn]

I didn't write tests. The few lines I run in the rails console are :

> file = File.open "file.png"
> f = Fichier.new :fichier => file
> f.save
> f.fichier.url
NoMethodError: undefined method `original_filename' for #<CarrierWave::Storage::Fog::File:0xab4134c>
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/processing/mime_types.rb:46:in `set_content_type'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `block in with_callbacks'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `each'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `with_callbacks'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/store.rb:96:in `retrieve_from_store!'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/versions.rb:232:in `block in retrieve_versions_from_store!'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/versions.rb:232:in `each'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/versions.rb:232:in `retrieve_versions_from_store!'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `block in with_callbacks'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `each'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/callbacks.rb:18:in `with_callbacks'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/uploader/store.rb:96:in `retrieve_from_store!'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/mount.rb:311:in `uploader'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/mount.rb:169:in `fichier'
from /var/lib/gems/1.9.1/gems/carrierwave-0.5.8/lib/carrierwave/mount.rb:157:in `fichier'
from (irb):5
from /var/lib/gems/1.9.1/gems/railties-3.1.0/lib/rails/commands/console.rb:45:in `start'
from /var/lib/gems/1.9.1/gems/railties-3.1.0/lib/rails/commands/console.rb:8:in `start'
from /var/lib/gems/1.9.1/gems/railties-3.1.0/lib/rails/commands.rb:40:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'

The weird behavior I noticed is that if I run the same line another time, I have the good result without error :

> f.fichier.url
=> "https://my-bucket.s3.amazonaws.com/uploads/file.png"

I see a method content_type in the S3::File module which use the HTTP header to define the file's content-type. So can I use this to solve my problem ? and how ?

Thank you :)

[gzigzigzeo]

At carrierwave/lib/carrierwave/processing/mime_types.rb:46 you can see this lines of code:

  if override || file.content_type.blank? || file.content_type == 'application/octet-stream'
      new_content_type = ::MIME::Types.type_for(file.original_filename).first.to_s 

I call set_content_type with override set to true. It could be that if S3 storage is used there is no need to override retreived content type. I do not really remember why did it called with override flag. You could make a fork, change line 8 of lib/carrierwave-meta/meta.rb from set_content_type(true) to set_content_type without args and run specs to see that everything is okay with Filesystem storage.

I think, it may solve your problem. I can try this thing for you and write specs for S3 storage at the weekend. You may try this thing yourself, or wait if you can.

[ndemoreau]

Hi, I try this but the problem is still there. ;-)

[manzhikov]

Hi,
I have same problem.
Do you have any updates?

[ndemoreau]

No. I rolled back to Paperclip...

[manzhikov]

I have solved. But it is monkey patch and works for me:

1. Migration:

class AddContentTypeAndFileSizeToAvatars < ActiveRecord::Migration
    def change
        add_column :avatars, :content_type, :string, default: '', null: false
        add_column :avatars, :file_size, :string, default: '0', null: false
    end
end

2. Added callback to model:

before_save :update_image_attributes

private
def update_image_attributes
    if image.present? && image_changed?
        self.content_type = image.file.content_type
        self.file_size = image.file.size
    end
end

3. And forked gem here:
   https://github.com/IldarManzhikov/carrierwave-meta

[clemens]

I've also just run into this issue. The simplest fix could probably be done in the CarrierWave gem itself by adding an original_filename method to Fog::File:

def original_filename
  ::File.basename(path)
end

[evie404]

I just ran into this, and the way I solved this was by skipping the content type step, since my app was only needed width and height information.

My fork is here in case anyone is interested:
https://github.com/rickypai/carrierwave-meta

[geemus]

I'm not familiar enough with this gem to comment too specifically, but the issue appears to be that the content_type check is happening on the wrong object. Carrierwave internally expects that you would be doing this on a CarrierWave::SanitizedFile object, not a CarrierWave::Storage::Fog::File. SanitizedFile defines the method you want/need so I think that is the right fix here. I wonder if perhaps the delegator stuff is what is causing it to refer to the wrong object type? Hope that helps.

[glebm]

@clemens Thanks for the monkey-patch!

[gzigzigzeo]

Fixed, monkey-patch does not needed anymore.

[jpayne]

Still happening to me in 0.0.5.

[nfriend21]

still happening to me too.

[torstenb]

Also just ran into this issue.

[elliot-nelson]

Also experiencing this issue running fog 1.28.0, carrierwave 0.10.0, and carrierwave-meta 0.0.5.

It looks like this round of the issue is actually in carrierwave -- pulling from the master branch fixes the issue. I'm sure this will come out as the stable tagged version in the near future, but in the meantime, just edit your Gemfile as follows:

gem 'carrierwave', github: 'carrierwaveuploader/carrierwave'