Defaults values for embedded documents with subclasses no longer with on 0.15 about mongomapper

[a2ikm]

Thank you for your reproducible code!
I found that it puts "OK" with mongo_mapper 0.15.0 to 0.15.3, but doesn't with 0.15.4 or higher.

So, how about upgrading to mongo_mapper 0.15.3 for now?
We need mongo_mapper 0.15.4+ for ruby 3.0+ and rails 7.0+.
If you don't use them, mongo_mapper 0.15.3 is good for your upgrade path I think.

On the other hand, I will investigate the error to fix it.

How I confirmed it

I changed the gemfile like:

gemfile do
  source 'https://rubygems.org'

  gem 'mongo_mapper', ARGV[0]
  gem 'activesupport', '~> 6.0.0'
end

and ran it with 0.15.x:

$ ruby a.rb 0.15.0           
OK
$ ruby a.rb 0.15.1
OK
$ ruby a.rb 0.15.2
OK
$ ruby a.rb 0.15.3
OK
$ ruby a.rb 0.15.4
/Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:467:in `block in initialize_default_values': undefined method `key?' for #<C _id: BSON::ObjectId('62d9cad23d572c2e3f265845'), _type: "C"> (NoMethodError)

          if !(except && except.key?(key.name))
                               ^^^^^
Did you mean?  keys
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:466:in `each'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:466:in `initialize_default_values'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:275:in `initialize_from_database'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:123:in `load'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:113:in `from_mongo'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys/key.rb:64:in `get'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:448:in `internal_write_key'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:468:in `block in initialize_default_values'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:466:in `each'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:466:in `initialize_default_values'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/keys.rb:267:in `initialize'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/sci.rb:77:in `initialize'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/callbacks.rb:8:in `block in initialize'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-6.0.5.1/lib/active_support/callbacks.rb:101:in `run_callbacks'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/embedded_callbacks.rb:78:in `run_callbacks'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/callbacks.rb:8:in `initialize'
        from /Users/masato/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/mongo_mapper-0.15.4/lib/mongo_mapper/plugins/partial_updates.rb:18:in `initialize'
        from a.rb:25:in `new'
        from a.rb:25:in `<main>'

[a2ikm]

I found that 7685eda made the issue.

I put a test script named a.rb, boot moongo, run git bisect as follows, and got it.

$ docker run -itd --publish 27017:27017 --rm mongo 
$ git bisect start v0.15.4 v0.15.3
$ git bisect run a.rb
: 
7685eda63631a774c3553260f47eae4539a2b6e0 is the first bad commit
:

a.rb is:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'

  gem 'mongo_mapper', path: "."
  gem 'activesupport', '~> 6.0.0'
end

class B
    include MongoMapper::EmbeddedDocument
end

class C < B
end

class A
  include MongoMapper::Document
  key :bar, B, default: -> {C.new}
end


MongoMapper.setup({'development' => {'uri' => 'mongodb://localhost:27017/test'}}, 'development', logger: Logger.new(File::NULL))

a = A.new

puts 'OK'

[a2ikm]

@Antsiscool
I made a patch: #711
If you have a time, could you try it?

[Antsiscool]

@a2ikm Thanks for making a patch. I will have a look and see if it fixes the problems I was seeing.

[a2ikm]

Thank you 😄

[Antsiscool]

@a2ikm The linked pull request did fix the problem with defualt values for embedded documents with subclasses.

However there were some other problems that it did not fix.

We also have some custom classes that specify to_mongo and from_mongo. e.g. TimeOfDay that is stored as the number of seconds since midnight in the database. Default values with the objects was still broken with the linked pull request.

I had a look at the 7685eda

I made the following monkey patch and it fixed the original problem and the issue with default values on custom objects.

module MongoMapper
  module Plugins
    module Keys

      class Key
        def default_value
          return unless default?

          if default.instance_of? Proc
            type ? type.to_mongo(default.call) : default.call
          else
            # Using Marshal is easiest way to get a copy of mutable objects
            # without getting an error on immutable objects
            type ? type.to_mongo(Marshal.load(Marshal.dump(default))) : Marshal.load(Marshal.dump(default))
          end
        end
      end
    end
  end
end

[a2ikm]

@Antsiscool
Thank you for your report and the patch!

Could you give me a reproducible code?
There is a test case on a custom class for a default value, and it has passed on the pull request. So I wonder how the error on your case happens.
https://github.com/mongomapper/mongomapper/blob/master/spec/functional/document_spec.rb#L96-L114

Regarding your patch, adding to_mongo to Key#default_value changes its return value for an embedded document key:

• current: Key#default_value returns an instance of the embedded document
• updated: Key#default_value returns an attributes hash of an instance of the embedded document

@smtlaissezfaire
Do you have any consideration on adding to_mongo to Key#default_value?
Checking nil of type like @Antsiscool's patch will be a workaround for #679.
Even if so, it changes a return value of Key#default_value as I wrote above.

[Antsiscool]

@a2ikm I have created a pull request that fixes the problem and adds some test cases.

[a2ikm]

@Antsiscool
Thank you for your patch and the new test cases!

I've investigated the current implemantation and found that from_mongo can be called with primitive values which are like integers stored in MongoDB, and application-side values including an instance of a custom class. The latter parameter is from Key#default_value in Keys#initialize_default_values.

mongomapper/lib/mongo_mapper/plugins/keys.rb

Line 480 in cb5267a

internal_write_key key.name, key.default_value, false

The custom class WindowSize used in the existing test case considers both of them, but TimeOfDay doesn't.

mongomapper/spec/support/models.rb

Line 10 in cb5267a

value.is_a?(self) ? value : WindowSize.new(value)

So, how about changing your implementation like the following based on #711?

def self.from_mongo(value)
  return nil if value.blank?

  value.is_a?(self) ? value : new(value.to_i)
 end

I think we shouldn't change the return value of Key#default_value by versions.
(It is undocumented and has been changed several times, so this can be controversial, though)

By the way, the latter case seems tricky against its name. I mean, from_mongo should be called only with primitive values. It would be incompatible and should be discussed in another issue.

[a2ikm]

By the way, the latter case seems tricky against its name. I mean, from_mongo should be called only with primitive values. It would be incompatible and should be discussed in another issue.

I tried to call from_mongo only with primitive values, but gave up:

• we need to convert a given default value to a given type with from_mongo for this test case:

  mongomapper/spec/functional/document_spec.rb

  Line 140 in cb5267a

  doc.foo.should == :'123'

• even if the type of a value matches with its key's type, we can't avoid the conversion. It is because the actual type can be different, e.g., a Hash typed value is actually an instance of HashWithIndifferentAccess:

  mongomapper/lib/mongo_mapper/extensions/hash.rb

  Line 9 in cb5267a

  HashWithIndifferentAccess.new(value || {})

[a2ikm]

@Antsiscool
I changed my mind and merged your pull request #713.

In addition, I created another pull request #715 to resolve my consideration I wrote above:

I think we shouldn't change the return value of Key#default_value by versions.
(It is undocumented and has been changed several times, so this can be controversial, though)

Its change won't discourage your expectation. If you had a time, please try it with the master branch.
Thank you!

[a2ikm]

This issue has been fixed since v0.16.0.
https://rubygems.org/gems/mongo_mapper/versions/0.16.0