rubiety / nilify_blanks

Provides a framework for saving incoming blank values as nil in the database in instances where you'd rather use DB NULL than simply a blank string.

<img src=“https://secure.travis-ci.org/rubiety/nilify_blanks.svg?branch=master” alt=“Build Status” />

Nilify Blanks¶ ↑

In Rails when saving a model from a form and values are not provided by the user, an empty string is recorded to the database instead of a NULL as many would prefer (mixing blanks and NULLs can become confusing). This plugin allows you to specify a list of attributes (or exceptions from all the attributes) that will be converted to nil if they are blank before a model is saved.

By default, columns set as NOT NULL will not be nilified, since that value cannot be persisted, and it might be better to catch that with a validation. If you want to nilify those columns anyway, you can use ‘only:` (see below) to explicitly mention them.

Only attributes responding to blank? with a value of true will be converted to nil. Therefore, this does not work with integer fields with the value of 0, for example. Usage is best shown through examples:

Requirements¶ ↑

As of v1.4.0, this gem requires Rails 4 and Ruby 2.2 or higher.

Install¶ ↑

Include the gem using bundler in your Gemfile:

gem "nilify_blanks"

Basic Usage¶ ↑

# Checks and converts all content fields in the model
class Post < ActiveRecord::Base
  nilify_blanks
end

# Checks and converts only text fields in the model
class Post < ActiveRecord::Base
  nilify_blanks types: [:text]
end

# Checks and converts only the title and author fields
class Post < ActiveRecord::Base
  nilify_blanks only: [:author, :title]
end

# Checks and converts all fields except for title and author
class Post < ActiveRecord::Base
  nilify_blanks except: [:author, :title]
end

# Checks and converts any fields, regardless of their null constraint
class Post < ActiveRecord::Base
  nilify_blanks nullables_only: false
end

Global Usage¶ ↑

You can also apply nilify_blanks to all models inheriting from ActiveRecord::Base:

ActiveRecord::Base.nilify_blanks

Or perhaps just a model namespace base class:

Inventory::Base.nilify_blanks

Specifying a Callback¶ ↑

Checking uses an ActiveRecord before_validation callback by default, but you can specify a different callback with the :before option. Any callback will work - just first remove the “before_” prefix from the name.

class Post < ActiveRecord::Base
  nilify_blanks before: :create
end

class Post < ActiveRecord::Base
  nilify_blanks before: :validation_on_update
end

RSpec Matcher¶ ↑

First, include the matchers:

require "nilify_blanks/matchers"

To ensure for a given column:

describe City do
  it { should nilify_blanks_for(:name) }
end

To ensure for all applicable content columns:

describe City do
  it { should nilify_blanks }
end

You can optionally match on options also:

describe City do
  it { should nilify_blanks_for(:name, before: :create) }
end

describe City do
  it { should nilify_blanks(before: :create) }
end

Running Tests¶ ↑

This gem uses appraisal to test with different versions of the dependencies. See Appraisal first for which versions are tested, then run to test all appraisals:

$ rake appraisal install
$ rake appraisal test

nilify_blanks's Issues

Impossible to override default settings

Using it, I stumbled over what looks like a major issue:

# initializer
ActiveSupport.on_load(:active_record) { ActiveRecord::Base.nilify_blanks }

# some class
class Post < ActiveRecord::Base
  nifily_blanks except: %i(citation) # Never taken into account.
end

I think we should embrace the rails callbacks way:

Calling the same callback multiple times will overwrite previous callback definitions.

What do you think?

---

Another note, using nilify_blanks will remove existing callbacks. Shouldn't we prevent this? Or at least warn users.

Add an option for trimifying as well

In other words, empty strings will become nil, but a string like " foo " will become "foo". Probably something like:

nilify_blanks :trimify => true

Deprecation warnings for rspec 3

Maybe the code could be structured to avoid these?

Deprecation Warnings:

`failure_message_for_should_not` is deprecated. Use `failure_message_when_negated` instead. Called from bundler/gems/nilify_blanks-36503fb06364/lib/nilify_blanks/matchers.rb:36:in `block in <top (required)>'.

`failure_message_for_should` is deprecated. Use `failure_message` instead. Called from 
bundler/gems/nilify_blanks-36503fb06364/lib/nilify_blanks/matchers.rb:32:in `block in <top (required)>'.

Seems to ignore column named `text`

We have a model with a column named text.

When we do just nilify_blanks, this column is not actually nilified.

When we do nilify_blanks only: [ :text ], it is nilified.

Could it be that this specific column name is treated differently?

In other models, with other column names (such as body_sv), the nilification works even when we don't declare the specific columns.

This is Rails 4.1.14.2 and nilify_blanks 1.2.1.

/ @Lavinia and @henrik

Release 1.3.0 to gem repository

It looks like the version was bumped to 1.3.0 a while ago, but a release was never made to rubygems. Please consider releasing.

Master is Pointing at 1.0.4

The version specified in version.rb on master is pointing at 1.0.4 while the current release appears to be 1.1.0. I'm not sure if something got overwritten at some point or 1.1.0 was tagged from a non-master branch, but you should probably conduct an audit before you cut a new version to see if there are any differences between 1.1.0 and master.

Ignore non-text fields

self.nilify_blanks_columns = self.content_columns.select(&:null).map(&:name).map(&:to_s)

Any reason why this select all columns? Doesn't this problem only apply to string, text, and perhaps hstore? Seems like saving an empty string to integer and date columns gives NULL.

warning: instance variable @_nilify_blanks_options not initialized

/Users/matthew.spence/.rvm/gems/ruby-2.2.3/gems/nilify_blanks-1.2.1/lib/nilify_blanks.rb:21: warning: instance variable @_nilify_blanks_options not initialized
/Users/matthew.spence/.rvm/gems/ruby-2.2.3/gems/nilify_blanks-1.2.1/lib/nilify_blanks.rb:53: warning: instance variable @_nilify_blanks_options not initialised

RSpec matchers

I think it will be great to make matchers for this gem, like gem 'shoulda'

describe City do
  it { should validate_presence_of(:name) } # self-explanatory
  it { should validate_uniqueness_of(:name) }
end

Smth. like that:

describe City do
  it { should nilify_blank } # or,
  it { should nilify_blank(:name) } # or,
  it { should nilify_blank(:name, on: :create) }
end

I can make this feature, but not so fast. I can do that till end of january.

Matcher "nilify_blanks" doesn't filter columns by type, causes failure

The nilify_blanks matcher runs this check to ensure the list of columns to nilify is correct:

model_class.nilify_blanks_columns == model_class.content_columns.select(&:null).map(&:name).map(&:to_s)

However, nilify_blanks_columns when initialised without any explicit columns loads the column list like so:

self.nilify_blanks_columns = self.content_columns.select(&:null).select {|c| options[:types].include?(c.type) }.map(&:name).map(&:to_s)

The issue then is that the matcher is NOT filtering by column types like the initialiser is. In my case, my model has a date field that is not included in nilify_blanks_columns (because the default types are only :string and :text) so my spec fails.

My temporary workaround, which is fine because I'd like everything to be NULL, is initialising with:

ActiveRecord::Base.nilify_blanks types: [:string, :text, :date, :time, :datetime, :inet]

Breaks rake db:migrate

nilify_blanks.rb:24

When trying to db:migrate. Commenting out nilify_blanks solves the issue.

Lazy-Evaluating Table Name

Since this doesn't lazy-evaluate table naming, it doesn't really work on an STI base class (or on ActiveRecord::Base). This should probably be redone to support this.

Please, add gem releases to GitHub too (to watch the diff between them)

Please, add GitHub releases too or just tag them. It is important to watch the diffs. Sometimes it is crucial when deciding to upgrade or not.

Cannot call rspec nilify_blanks on STI models

I changed one of my tables into an STI table so I could subclass my Post model into Article, Review and MicroBlog. Immediately, the following test started failing:

describe "nilify_blanks" do
  it { is_expected.to nilify_blanks(before: :validation) }
end

This despite the following in my ApplicationRecord class:

class ApplicationRecord < ActiveRecord::Base
  ...
  nilify_blanks before: :validation
  ...
end

At first I thought it was because of some Rails internal doing something strange to the type attribute because it's STI. But then I discovered this works:

describe "nilify_blanks" do
  # Must specify individual fields for STI models.
  it { is_expected.to nilify_blanks_for(:alpha,   before: :validation) }
  it { is_expected.to nilify_blanks_for(:body,    before: :validation) }
  it { is_expected.to nilify_blanks_for(:slug,    before: :validation) }
  it { is_expected.to nilify_blanks_for(:summary, before: :validation) }
  it { is_expected.to nilify_blanks_for(:title,   before: :validation) }
  it { is_expected.to nilify_blanks_for(:type,    before: :validation) }
end

So I can use the nilify_blanks_for matcher on the STI type field and every other text/string field in the model. But when I use the nilify_blanks matcher, boom.

This is the failure before my change:

1) Post concerns behaves like an_application_record included nilify_blanks should nilify blanks [:before, :validation] and [:types, [:string, :text, :citext]]
   Failure/Error: it { is_expected.to nilify_blanks(before: :validation) }
     expected to nilify blanks {:before=>:validation, :types=>[:string, :text, :citext]}

This is a bummer because I went from being able to test nilify_blanks in a shared example for ApplicationRecord that's included in every model spec, to having to test it individually in just this one random model. The nilify_blanks matcher still works on all my non-STI models.

Any thoughts?

Thanks so much for an amazing gem that I use on every project.

Calling ActiveRecord::Base.nilify_blanks in an initializer does not seem to work

If I create a file in config/initializers with

ActiveRecord::Base.nilify_blanks

Then no effect.

But placing this line:

  nilify_blanks

Below the declaration of my model seems to work fine. I'm using all very current gems, and the master tip of this project...

Bug in define_nilify_blank_methods

In file lib/nilify_blanks.rb

Line 54:
return if @nilify_blank_methods_generated

Line 78:
@nilify_blanks_methods_generated = true

Is line 54 missing an 's'

Rails 4 support

I am working to upgrade my application to Rails for, and I get this error when trying to run RSpec:

/gems/activerecord-4.0.0/lib/active_record/dynamic_matchers.rb:22:in `method_missing': undefined local variable or method `nilify_blanks' for #<Class:0x007fb59b095a08> (NameError)

Using Ruby 2.0.0, Rails 4.0.0

Renaming the `master` branch to `main`

Here's why GitHub renamed its master branch to main.

Support for postgres array and hstore columns

Example array attribute:
:column => ["", ""]
should result in nil.
:column => ["hello", ""]
should result in ["hello"].

Nested Forms

Nilify_blanks not working with nested form I'm afraid.

Global use

Is there an easy way to use this on all models automatically?

Perhaps an initializer?

class ActiveRecord::Base
    nilify_blanks
end

Make nilify_blanks more generic and applicable even for non-DB

Hi,

Please consider adding the following features to nilify_blanks.

1. Add an 'after' option similar to that of 'before'
2. Allow it to be used by the more general ActiveModel::Model instead of just ActiveRecord::Base

The reasoning is that it can be useful to apply nilify_blanks in after_initialize, so that we can treat all attributes consistently (not only ones that are saved into DB, but also ones that are not yet saved, and even ones that are not DB columns).

It's true that the initial intent was to provide consistency for stored DB values. But it's better if this consistency also applies at code (ruby) level, so we know that unspecified values are always nil.

This consistency can then be taken to a higher level, whereby non-DB-backed models (ActiveModel::Model) also have the same behaviour.

Thank you

Consider changing the default hook to before_validation

We ran into a unexpected issue in our Rails application that could have been avoided with a different default value in nilify_blanks.

Here's a (very) trimmed down version of one of our models:

class Lab < ActiveRecord::Base
  nilify_blanks

  validates_uniqueness_of :title, scope: [:version_string]
end

Because nilify_blanks hooks into before_save by default, the following lines will create two labs:

lab1 = Lab.create(title: "My title", version_string: nil)
lab2 = Lab.create(title: "My title", version_string: '')

Practically speaking, we saw this when users would clear the version string on a lab from a form in our UI:

1. A lab starts with a version string set (like "1.0")
2. In the UI, someone clears the version string field
3. We attempt to update a lab with a version string of ""
4. Our validation passes because "" != nil
5. Then the empty string turns into a nil, the lab saves as if nothing went wrong, and we have data problems now

The above can be avoided by using nilify_blanks before: :validation instead (which we're using now). With that said, would it make sense to make that the default for nilify_blanks going forward or is there a good reason that the current hook is on before_save? I realize changing this would probably be a breaking change at this point.

ActiveRecord::Dirty methods do not use nilify_blanks logic

Not sure if this is pushing too far, but I get incorrect behavior when trying to use methods like changed? and changes when nilify_blanks is applied to a model.

For example:

--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
email:
- 
- ''
notes:
- 
- ''

I had to configure nilify_blanks like this:

nilify_blanks before: :validation

And then do this clunky logic to work around this problem:

# Save referral. We want to track if anything changed to decide how to treat any comments below.
referral.attributes = referral_attributes
# Force nilify_blanks to do its thing before validation so we can check for the true changes.
referral.valid?
# See if there were changes before updating the referral record. (This will always be `false` after we run
# `update`.)
referral_changed = referral.changed?
referral.save

Not sure how involved it would be to fix this or if it would be appropriate to do so. Or is there a better way to do what I'm trying to accomplish?

Availability on RubyGems

Would be great if we could:
sudo gem install nilify_blanks

Thread safety

Hi,

You're currently redefining define_attribute_methods like this:

    def define_attribute_methods
      if super
        define_nilify_blank_methods
      end
    end

It's not thread safe; two threads could end up trying to define the before_save callback on the class.

In AR, the contents of define_attribute_methods is wrapped in a mutex:

https://github.com/rails/rails/blob/master/activerecord/lib/active_record/attribute_methods.rb#L74-L85

I think you could just reuse the same mutex.

Cheers

Not working for fields with _id suffix?

We have a model Team that contains a t.string "stripe_customer_id" field that references a Customer object on https://stripe.com.

We added nilify_blanks like this:

class Team < ActiveRecord::Base
  nilify_blanks
end

However nilify_blanks seems to work for all fields EXCEPT stripe_customer_id:

(0.1ms)  BEGIN
  SQL (0.6ms)  UPDATE "teams" SET "stripe_customer_id" = '', "billing_address_country" = NULL, "updated_at" = '2015-11-18 14:16:43.918496' WHERE "teams"."id" = 182  [["stripe_customer_id", ""], ["billing_address_country", nil], ["updated_at", Wed, 18 Nov 2015 14:16:43 UTC +00:00], ["id", 182]]
(0.4ms)  COMMIT

Is there a special handling for fields that end on _id ?

nilify also json/jsonb values/columns?

In my models I use to set a callback to nilify blank json/jsonb values (and the whole column if it's just empty), in the following way:

before_save :nilify_blank_stored_attributes

  def nilify_blank_stored_attributes
    self.class.stored_attributes.keys.each do |stored_attribute|
      send(stored_attribute).delete_if{|k,v| v.blank?}
      send(stored_attribute.to_s.concat('=').intern, nil) if send(stored_attribute).empty?
    end
  end

How about adding this feature?

Not compatible/poor performance with Rails 4.0.4+ and and Rails 4.1.0+

I recently made the upgrade to Rails 4.0.4 from Rails 4.0.3 and noticed extremely degraded performance (unicorns workers started timing out). I rolled back once I traced it to the upgrade and started profiling in a test environment.

I traced the performance degradation to NilifyBlanks::ClassMethods#define_attribute_methods which is used to hook into ActiveRecord::AttributeMethods::ClassMethods#define_attribute_methods.

Previously, in Rails 4.0.3 and below, ActiveRecord calls #define_attribute_methods only if they have not yet been generated yet in ActiveRecord::AttributeMethods::ClassMethods#method_missing (https://github.com/rails/rails/blob/v4.0.3/activerecord/lib/active_record/attribute_methods.rb#L146) and ActiveRecord::AttributeMethods::ClassMethods#respond_to? (https://github.com/rails/rails/blob/v4.0.3/activerecord/lib/active_record/attribute_methods.rb#L191).

However, in Rails 4.0.4+ and Rails 4.1.0 this logic was completely moved into #define_attribute_methods for a better thread safe implementation (see rails/rails@30efdd6).

As a result, nilify_blanks initializes itself through NilifyBlanks::ClassMethods#define_nilify_blank_methods (https://github.com/rubiety/nilify_blanks/blob/master/lib/nilify_blanks.rb#L10) every time method_missing or respond_to is called on a ActiveRecord model and results in a huge performance issue.

I'd be glad help put in a PR to work around the issue and saw that you used to check for table existence before hooking into define_attribute_methods in #8 . Is there a way we can load the nilify_blanks code without regressing to the method used before?