citusdata / activerecord-multi-tenant

Rails/ActiveRecord support for distributed multi-tenant databases like Postgres+Citus

activerecord-multi-tenant

Introduction Post: https://www.citusdata.com/blog/2017/01/05/easily-scale-out-multi-tenant-apps/

ActiveRecord/Rails integration for multi-tenant databases, in particular the open-source Citus extension for PostgreSQL.

Enables easy scale-out by adding the tenant context to your queries, enabling the database (e.g. Citus) to efficiently route queries to the right database node.

Installation

Add the following to your Gemfile:

gem 'activerecord-multi-tenant'

Supported Rails versions

All Ruby on Rails versions starting with 6.0 or newer (up to 7.0) are supported.

This gem only supports ActiveRecord (the Rails default ORM), and not alternative ORMs like Sequel.

Usage

It is required that you add multi_tenant definitions to your model in order to have full support for Citus, in particular when updating records.

In the example of an analytics application, sharding on customer_id, annotate your models like this:

class PageView < ActiveRecord::Base
  multi_tenant :customer
  belongs_to :site

  # ...
end

class Site < ActiveRecord::Base
  multi_tenant :customer
  has_many :page_views

  # ...
end

and then wrap all code that runs queries/modifications in blocks like this:

customer = Customer.find(session[:current_customer_id])
# ...
MultiTenant.with(customer) do
  site = Site.find(params[:site_id])
  site.update! last_accessed_at: Time.now
  site.page_views.count
end

Inside controllers you can use a before_action together with set_current_tenant, to set the tenant for the current request:

class ApplicationController < ActionController::Base
  set_current_tenant_through_filter # Required to opt into this behavior
  before_action :set_customer_as_tenant

  def set_customer_as_tenant
    customer = Customer.find(session[:current_customer_id])
    set_current_tenant(customer)
  end
end

Rolling out activerecord-multi-tenant for your application (write-only mode)

The library relies on tenant_id to be present and NOT NULL for all rows. However, its often useful to have the library set the tenant_id for new records, and then backfilling tenant_id for existing records as a background task.

To support this, there is a write-only mode, in which tenant_id is not included in queries, but only set for new records. Include the following in an initializer to enable it:

MultiTenant.enable_write_only_mode

Once you are ready to enforce tenancy, make your tenant_id column NOT NULL and simply remove that line.

Frequently Asked Questions

• What if I have a table that doesn't relate to my tenant? (e.g. templates that are the same in every account)

  We recommend not using activerecord-multi-tenant on these tables. In case only some records in a table are not associated to a tenant (i.e. your templates are in the same table as actual objects), we recommend setting the tenant_id to 0, and then using MultiTenant.with(0) to access these objects.

• What if my tenant model is not defined in my application?

  The tenant model does not have to be defined. Use the gem as if the model was present. MultiTenant.with accepts either a tenant id or model instance.

Credits

This gem was initially based on acts_as_tenant, and still shares some code. We thank the authors for their efforts.

License

Copyright (c) 2018, Citus Data Inc.
Licensed under the MIT license, see LICENSE file for details.

activerecord-multi-tenant's Issues

How to adds a partition_key for a table that already exits

Hello, I am trying to migrate my app to use CITUS. To start i am trying to update only County model that are linked to my tenant (Company):

# My "tenant model"
class CreateCompanies < ActiveRecord::Migration[5.2]
  def change
    create_table :companies do |t|
      t.string :name
      t.references :company_manager, foreign_key: true

      t.timestamps
    end
  end
end

# Initial Migration
class CreateCounties < ActiveRecord::Migration[5.2]
  def change
    create_table :counties do |t|
      t.string :name
      t.references :county_supervisor, foreign_key: true
      t.references :company

      t.timestamps
    end
  end
end

# Model
class County < ApplicationRecord
  multi_tenant :company
  belongs_to :company
   ...
end

This models already exits in my database, how i can add partition key do counties table and integrate to citus?

undefined method `wrap_methods' for MultiTenant:Module (NoMethodError) Did you mean? methods

I am using version 2.1.5 with Rails version 6.1.7.

When I am trying to start up the server, I get the following error:

Traceback (most recent call last):
	67: from bin/rails:3:in `<main>'
	66: from bin/rails:3:in `load'
......
	 1: from /Users/yang/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-6.1.7/lib/active_support/lazy_load_hooks.rb:71:in `class_eval'
/Users/yang/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-multi-tenant-2.1.5/lib/activerecord-multi-tenant/model_extensions.rb:134:in `block in <main>': undefined method `wrap_methods' for MultiTenant:Module (NoMethodError)
Did you mean?  methods

Ruby 2.7 - Last argument as keyword parameters is deprecated

When upgpraded to Ruby 2.7 there is some new syntax to be adopted:

activerecord-multi-tenant-1.0.4/lib/activerecord-multi-tenant/model_extensions.rb:57: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

Update with string argument cause error

Hi. Thank you for nice gem.

We are using activerecord-multi-tenant with mysql.

AbstractMysqlAdapter executes some commands by update method with string argument.

ex: disable_referential_integrity

      def disable_referential_integrity #:nodoc:
        old = query_value("SELECT @@FOREIGN_KEY_CHECKS")

        begin
          update("SET FOREIGN_KEY_CHECKS = 0")
          yield
        ensure
          update("SET FOREIGN_KEY_CHECKS = #{old}")
        end
      end

With rails 5.2 or higher, acitiverecord-multi-tenant wrapping up update method and it looks not to expect called with string argument but just arel objects. Raise expection "undefined method `ast' for string".

Could modify it to accept string arugment? maybe just let it through.

Automate the Citus setup by enhancing the Rails rake tasks

So currently, in order for Citus to work, one needs to do the following steps:

• install the Citus package;
• create n Postgres clusters (1 coordinator + (n-1) workers);
• tell each cluster about Citus by adding "shared_preload_libraries = 'citus'" to its respective postgresql.conf
• create the extension in each cluster;
• tell the coordinator about the workers.

So basically, I've described the steps from the guide for OS X. Now, the problem with this is that one cannot just reset (drop/create) the db anymore via rake db:reset, as these steps will be lost. Even worse, when running rake db:migrate RAILS_ENV=test, is dumps a version of the schema, which doesn't contain the Citus extension and thus one could accidentally commit it.

So my idea is - can't we augment the rake tasks for interacting with the DB in such a way, that some of the steps, described above, happen automatically. For example, in the project I am currently working on, I've added the following rake task:

namespace :db do
  desc "Enables Citus extension in the database"
  task enable_citus: :environment do
    ActiveRecord::Base.connection.execute('CREATE EXTENSION citus;')
  end
end

Rake::Task['db:create'].enhance do
  Rake::Task['db:enable_citus'].invoke
end

Now, I don't observe the described side-effect of running migrations in the test database anymore.

What do you think of making similar things part of the gem? For example, I was thinking of adding the ports for the coordinator and workers as some kind of configuration to Rails and then also enhancing rake db:create with something, which executes "SELECT * from master_add_node('localhost', port_number);" for each port.

Add Citus and Postgres Compatibility matrix for tests

Right now, tests are being executed for citus 7.5.1 which is a very old version and there are no tests for various postgres versions.
These tests should be added

Puma Multi thread support

Do this gem supports Multi threaded servers like puma ?

CollectionProxy.find raises error if block is given

version: 2.1.1

CollectionProxy.find must be aliased to CollectionProxy.to_a.find as below.
https://github.com/rails/rails/blob/main/activerecord/lib/active_record/relation/finder_methods.rb#L67

However, with activerecord-multi-tenant, CollectionProxy.find raises an error if block is given.
Below may be the cause.

activerecord-multi-tenant/lib/activerecord-multi-tenant/model_extensions.rb

Line 143 in e1d50b4

MultiTenant.wrap_methods(ActiveRecord::Associations::CollectionProxy, '@association.owner', :find, :last, :take, :build, :create, :create!, :replace, :delete_all, :destroy_all, :delete, :destroy, :calculate, :pluck, :size, :empty?, :include?, :equals_mt, :records, :append_mt, :find_nth_with_limit, :find_nth_from_last, :null_scope?, :find_from_target?, :exec_queries)

Thanks.

Travis CI integration

We should have automated tests for all the supported Rails versions.

Add `MultiTenant.without` to disable tenants for a block

Something like this would override the current_tenant for a block to for cross-tenant queries on an as-needed basis.

module MultiTenant
  def self.without(&block)
    return block.call if self.current_tenant.nil?
    old_tenant = self.current_tenant
    begin
      self.current_tenant = nil
      return block.call
    ensure
      self.current_tenant = old_tenant
    end
  end
end

This would enable something like:

MyModel.count
=> 50
MultiTenant.current_tenant = User.first
MyModel.count
=> 1
MultiTenant.without do
  MyModel.count
=> 50
end

Happy to submit a tested pull-request if the maintainers are interested.

Issue when having multiple databases with tables with the same name.

There is an issue when having multiple databases with tables with the same name.

The case I experienced is the following: I have 2 databases, the main postgres database that uses the citus extension, and a smaller sqlite db. There are some tables in both databases that have the same name.

I set up Active Record 6 to work with both databases and I use the multi-tenant extension in the main db. When I try to delete a record in the sqlite db the multi-tenant gem picks the model from the main db (that has a table with the same name) and inserts the tenant clause causing the query to fail.

I believe this comes from the fact that the multi tenant models are stored in a hash indexed by table name https://github.com/citusdata/activerecord-multi-tenant/blob/master/lib/activerecord-multi-tenant/multi_tenant.rb#L27-L30

Model.find not using correct Tenant, when logging with a new one

When i change a devise logged user that belongs from a tenant (user1.customer_id => 1) to another user of another tenant (user2.customer_id => 2), the query made from the Model.find method is confused and still uses the other tenant id.

Instead, for the index action, the Model.all works fine.

But more strange, is that if i restart the rails server and try again, then it works correctly.

(OBS: Is not a before_action problem, i try also to put the code directly on the show action, and happens the same bug)

The code i'm using is this:

class ApplicationController < ActionController::Base
  protect_from_forgery with: :null_session
  respond_to :json
  
  set_current_tenant_through_filter #Required to opt into this behavior
  before_action :set_customer_as_tenant

  def set_customer_as_tenant
    customer_id = user_signed_in? ? current_user.customer_id : false

    #  This log is always the correct logged user customer_id
    logger.debug "Current_user customer_id on application_controller set_customer_as_tenant: #  {customer_id}"

    set_current_tenant(customer_id)
  end
end

class UsersController < ApplicationController
  before_action :authenticate_user!
  before_action :set_user, only: [:show, :update, :destroy]
  
  def index
    #  This query works fine, using the correct customer_id as the actual tenant
    @users = User.all
  end
  
  def show
    #  This query: @user = User.find(params[:id]), works with the same bug directly here
  end

  private
  
  def set_user
    if params.has_key?(:id)

      #  This log is always the correct logged user customer_id
      logger.debug "Current_user customer_id on users_controller set_user: #  {current_user.customer.id}"
      
      @user_test = User.new()
      #  This log is always the correct logged user customer_id; so it can be supposed that the next query should be done with the correct customer_id (tenant)
      logger.debug "@user_test.customer_id on users_controller set_user: #  {@user_test.customer_id}"

      #  HERE IS THE BUG!
      @user = User.find(params[:id])
      #  Performs this query: User Load (0.4ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 AND "users"."customer_id" = 1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
      #  Where this condition: "users"."customer_id" = 1, is using the incorrect customer_id

      #  Instead, this fix the bug, but is not the idea while this should be the gem job
      #  @user = User.find_by!(id: params[:id], customer_id: current_user.customer.id)
    else
      @user = current_user
    end
  end

  def user_params
    params.require(:user).permit(:email, :password)
  end
end

class User < ApplicationRecord
  # Include default devise modules. Others available are:
  # :registerable, :confirmable, :lockable, :timeoutable and :omniauthable
  devise :database_authenticatable,
         :recoverable, :rememberable, :trackable, :validatable
  
  multi_tenant :customer
end

class Customer < ApplicationRecord
  has_many :users
  
  validates :name, presence: true, uniqueness: true
end

Multi-tenant produces incorrect where clauses when joining between a table with a custom partition_key and one without

More details can be found #150

Include original call-site in QueryMonitor

The next release is going to add a query monitor feature that hints you at queries that might be missing the tenant_id filter because there is no wrapping MultiTenant.with(..) block:

b048e43

It might be worthwhile to include the original call site, as proposed for Rails' own statement logging here: rails/rails#26815

Allow overriding primary key columns/ordering in ActiveRecord migrations

We currently are using version 1.0.2 with Postgres partitioned tables and have had to (re-)monkey patch the pkey generation for it to work.

Our shard key is user_id ("partition_key" in the ActiveRecord code) and the Postgres partition key is timestamp. To satisfy the uniqueness constraints of both our "space" of users, as well as the time of partitions, we need all of id, user_id, and timestamp.

Due to our use case and our relative lack of reliance on primary keys, we have opted for a compound primary key of (user_id, timestamp, id). It provides the uniqueness constraints, relatively OK filtering properties, as well as convenient indexes for range scans across time without us duplicating those. I imagine other cases involving partitions would still prefer to have the id as the first element of the key, but not all.

The proposal here is to allow some form of tuple override which the migrations helper will use when altering the pkeys. I would happy to discuss any other options and/or create a PR to upstream this functionality.

Sidekiq Documentation

Can one detail how you run Sidekiq jobs with AMT?

Digging through the code I found this:

https://github.com/citusdata/activerecord-multi-tenant/blob/c4747ade6e51259b1915f3984bea1e9ea3fc1fc4/lib/activerecord-multi-tenant/sidekiq.rb

which seem to mirror the acts_as_tenant code which is just as vague on how to actually implement this.

I tried this adding require 'activerecord-multi-tenant/sidekiq' to mu initializer then adding this to my job call:

Searchkick::ProcessQueueJob.perform_later(class_name: "Location", multi_tenant: { class: 'Company', company_id: 9 } )

Sidekiq jobs fail with:

ArgumentError: unknown keyword: multi_tenant

Make has_and_belongs_to_many relationships work when using Citus (include tenant_id)

Currently a query that uses has_and_belongs_to_many (habtm) will not pull in the tenant_id for the habtm table itself.

This happens even if the table includes the tenant_id as an additional column, due to the Rails/ActiveRecord query builder not knowing that it needs to be included at all times.

We should be able to patch up ActiveRecord to include the tenant_id correctly:

• Variant 1) Include an explicit habtm_table.tenant_id = $1 condition

• Variant 2) Include the tenant_id in the JOIN between the habtm_table and the target_table

Until this is fixed this can be worked around by creating an explicit model for the intermediate table.

NameError: uninitialized constant Arel::Visitors::DepthFirst

Got this error NameError: uninitialized constant Arel::Visitors::DepthFirst when updated to Rails 6.1, using the following version - gem "activerecord-multi-tenant", "~> 1.1"

Found similar issues: rgeo/rgeo-activerecord#55

Heroku instructions

Is the Heroku citus add on a dependency for Heroku apps?

Any instructions regarding Heroku deployments are welcome.

Thanks.

Wrong SQL syntax LEFT JOIN with ActiveStorage (syntax error at or near "AND") - Rails 6.1.1

Ruby 3.0.0
Rails 6.1.1

Having MultiTenant with ActiveStorage

# action.rb

class Action < ApplicationRecord
  multi_tenant :setting_tenant, class_name: 'Setting::Tenant'

  has_many_attached :files
end

Then when I run

MultiTenant.current_tenant = Setting::Tenant.first

Action.left_outer_joins(:files_attachments).to_sql

The SQL looks like

SELECT "actions".*
FROM "actions"
LEFT OUTER JOIN "active_storage_attachments" ON  AND "active_storage_attachments"."setting_tenant_id" = 1
  AND "active_storage_attachments"."record_type" = 'Action'
  AND "active_storage_attachments"."name" = 'files'
  AND "active_storage_attachments"."record_id" = "actions"."id"
  AND "actions"."setting_tenant_id" = 1
WHERE "actions"."setting_tenant_id" = 1"

as you can see the condition in LEFT JOIN starts with ON AND which should not be there and throws:

ActiveRecord::StatementInvalid (PG::SyntaxError: ERROR:  syntax error at or near "AND")
LINE 1: ... LEFT OUTER JOIN "active_storage_attachments" ON  AND "activ...

Referential Integrity causes foreign key errors with creation of fixtures

Adding the gem caused my test suite to fail due to foreign key index violations with fixture creations.
Simple fix is just to change referential_integrity.rb to:

# Workaround for https://github.com/citusdata/citus/issues/1080
# "Support DISABLE/ENABLE TRIGGER ALL on distributed tables"

require 'active_record/connection_adapters/postgresql_adapter'

module ActiveRecord
  module ConnectionAdapters
    class PostgreSQLAdapter < AbstractAdapter
      def supports_disable_referential_integrity?
        if Rails.env.test?
          true
        else
          false
        end
      end
    end
  end
end

Not sure if there are any unintended consequences...

NoMethodError: undefined method `relation`

ActiveRecord (>=6.0.3) and AR::MultiTenant cause "NoMethodError: undefined method relation" when joining the multitenant tables and ActiveStorage tables.

The error has been occured since rails/rails#38929 is merged.

Steps to reproduce the issue:

• Save this code to app.rb:

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  # Activate the gem you are reporting the issue against.
  gem "rails", ENV["RAILS_VERSION"] || "6.0.3"
  gem "sqlite3"
  gem "activerecord-multi-tenant", require: false
end

require "active_record/railtie"
require "active_storage/engine"
require "tmpdir"
require "activerecord-multi-tenant"

class TestApp < Rails::Application
  config.root = __dir__
  config.hosts << "example.org"
  config.eager_load = false
  config.session_store :cookie_store, key: "cookie_store_key"
  secrets.secret_key_base = "secret_key_base"

  config.logger = Logger.new($stdout)
  Rails.logger  = config.logger

  config.active_storage.service = :local
  config.active_storage.service_configurations = {
    local: {
      root: Dir.tmpdir,
      service: "Disk"
    }
  }
end

ENV["DATABASE_URL"] = "sqlite3::memory:"

Rails.application.initialize!

require ActiveStorage::Engine.root.join("db/migrate/20170806125915_create_active_storage_tables.rb").to_s

ActiveRecord::Schema.define do
  CreateActiveStorageTables.new.change

  create_table :tenants
  create_table :groups, force: true do |t|
    t.integer :tenant_id, null: false
  end
  create_table :users, force: true do |t|
    t.integer :tenant_id, null: false
    t.integer :group_id, null: false
  end
end

class Tenant < ActiveRecord::Base
end

class Group < ActiveRecord::Base
  multi_tenant :tenant
  has_many :users
end

class User < ActiveRecord::Base
  multi_tenant :tenant
  belongs_to :group

  has_one_attached :profile
end

require "minitest/autorun"

class BugTest < Minitest::Test
  def test_join
    tenant = Tenant.create!
    group = Group.create!(tenant: tenant)

    User.create!(
      tenant: tenant,
      group: group,
      profile: {
        content_type: "text/plain",
        filename: "dummy.txt",
        io: ::StringIO.new("dummy"),
      }
    )

    assert_equal group.id, Group.joins(users: { profile_attachment: :blob }).find(group.id).id
  end
end

• Compare the results of theese two runs:
  • RAILS_VERSION=6.0.2.2 ruby app.rb
  • RAILS_VERSION=6.0.3 ruby app.rb

Reloading association record with selective columns result in `ActiveModel::MissingAttributeError: missing attribute: tenant_id `

When using gems like graphql or similar, they can at time returns records from association which do not contain the partition_key column. In cases, we'd see ActiveModel::MissingAttributeError: missing attribute error raised. A very stripped down example is

Story.find(6).events.select(:id).last.reload
...

ActiveModel::MissingAttributeError: missing attribute: tenant_id

This is happening from the overloading here

activerecord-multi-tenant/lib/activerecord-multi-tenant/model_extensions.rb

Line 142 in a6c8c2d

MultiTenant.wrap_methods(ActiveRecord::Associations::Association, 'owner', :reload)

Disabling that line resolves the issue. I also removed that line (shayonj#1) and no specs were failing (outside of the setting up ruby ones).

The line raising the exception is

activerecord-multi-tenant/lib/activerecord-multi-tenant/multi_tenant.rb

Line 137 in a6c8c2d

MultiTenant.with(#{owner}.public_send(#{owner}.class.partition_key)) { #{original_method_name}(...) }

Very specifically: #{owner}.public_send(#{owner}.class.partition_key)

One potential fix we could do is

            begin
              MultiTenant.with(#{owner}.public_send(#{owner}.class.partition_key)) { #{original_method_name}(...) }
            rescue ActiveModel::MissingAttributeError
              #{original_method_name}(...)
            end

This would ensure that execution won't break and the output would be what it would've been originally

Story.find(6).events.select(:id).last.reload
...

 [#<Event:0x00000001144c6ef8 id: 6 ....>,

Access Tenant Globally? (i.e. in models etc.)

Is there no way to access the current tenant in models etc? Apartment could do this easy.

Validate uniqueness globally?

Hello 👋

I'm running into an issue with uniqueness validation. In this case, I want to make sure that something is unique across all tenants. But when the current tenant is set the validation rule only checks if the attribute is unique for that tenant. It works fine when no tenant is set. But the tenant will be set when a record is created during a request.
Here is an example:

This test passes:

This test fails:

It thinks the metric is valid even though a metric in the first organization already has this origin_id.

Is there a way around this?

I'm on 1.0.4.

Use CurrentAttribute instead of RequestStore

It has built-in support in Rails and also with Sidekiq and ActiveJob

Compatibility with ActiveAdmin?

Anyone managed to get this to work with ActiveAdmin? It seems that with AA you can only access the records that your normal user was last logged in under (i.e. AR is still scoping the DB calls with the last tenant). I know there is the MultiTenant.without do but it's not clear you can integrate that with AA.

I thought about adding a current_user.admin_user? to the model tenant definition but not sure if that is right or would work.

After updating to 2.1.1 or later, getting an error.

We are using activerecord-multi-tenant with MySQL.
We have updated from 2.0.0 to 2.1.3 and now we get ActiveRecord::StatementInvalid error.

I think this is due to the change in #156 .

There is no pg_extension table in MySQL, so ActiveRecord::Migration.citus_version will always give an error.

As a result of ActiveRecord::Migration.citus_version being called by ActiveRecord::SchemaDumper, we, as MySQL users, can no longer run rails db:schema:dump.

Is there any good workaround?

Regards,

prevent certain sub-domains from created by the user?

is it posible to prevent certain sub-domains from created by the user?

Thanks

MultiTenant affects automatic_inverse_of on associations

The following scenario will fail. When using belongs_to the association tries to automatically build the inverse_of, something about the way that belongs_to is defined in MultiTenant seems to affect this automatic building process.

class Account < ActiveRecord::Base
  multi_tenant :account
  has_many :projects
  has_one :manager
end

class Manager < ActiveRecord::Base
  multi_tenant :account
  belongs_to :project
end

 # Reflections
  describe 'with unsaved association' do
    before do
      @account = Account.create!(name: "reflection tenant")
      @manager = Manager.new(account: @account)
      MultiTenant.current_tenant = @account
      @account.update(name: "reflection tenant update")
    end

    it "persists the reflected association" do
      expect(@manager.persisted?).to eq(true)
    end
  end

This may affect other association types but the workaround currently if relying on this behavior is to do the following

class Account < ActiveRecord::Base
  multi_tenant :account
  has_many :projects
  has_one :manager, inverse_of: :account
end

class Manager < ActiveRecord::Base
  multi_tenant :account, inverse_of: :manager
  belongs_to :project
end

RSpec Validations - Failing due to MultiTenant::TenantIsImmutable:

I've just recently started to use activerecord-multi-tenant, and for my test suite, without the gem, the following code works as expected

expect(audio).to belong_to(:account)

Once I introduce the gem, I am getting the following errors for my tests:

  1) Media::Audio associations
     Failure/Error: expect(audio).to belong_to(:account)

     MultiTenant::TenantIsImmutable:
       MultiTenant::TenantIsImmutable

I am not sure what I am missing, or what is the correct path to fix this failure

Help is greatly appreciated

class variable is not defined if first model referenced is not multi tenant

I see this is resolved in master (commit bf97bf5). Could a new release be published?

ActiveRecord 6.1 removes `DepthFirst` visitor

Upcoming Raiils version 6.1 drops Arel::DepthFirstvisitor. This means applications using Rails > 6.0 crash on startup. The relevant PR is here: rails/rails#36492

Support belongs_to optional: true for easier rollouts for existing models

Rails 5+ supports belongs_to :model, optional: true, where the presence of the relation is optional. Having this as part of roll out on existing models is nice, since you can add the new association and multi_tenant definition on existing models without having to re-write most code paths in one go, but spread them out in smaller changes. Then, once the migration is complete, you can remove the optional: true and the NOT NULL on table.

However, if a model has a belongs_to :model, optional: true defined today, we lose that here

activerecord-multi-tenant/lib/activerecord-multi-tenant/model_extensions.rb

Line 58 in 8e2d5be

belongs_to tenant_name, **options.slice(:class_name, :inverse_of).merge(foreign_key: options[:partition_key])

Thus, making it hard to perform easy rollouts with optional: true.

Conflict with ActiveRecord.table_name - multi_tennat_id not included in query

When adding multi tenant to a model, and specifying table_name on same model it seems that the order of commands matter.

Here is and example of code in question, I can't share the real code (but can try to create a minimal reproduction if needed):

1. without tennat id in query

class MyModel
  multi_tenant :account #First tennat, then table name
  self.table_name = 'my_table_name'
end

class OtherModel
  has_one :some_model,
          class_name: 'MyModel'
end

#when executing
obj = OtherModel.find(...)
obj.some_model

• Produced query does not have account_id

SELECT "my_table_name".* FROM "my_table_name" WHERE "my_table_name"."some_other_prop" = 10 LIMIT 1

2. with tennat id in query

class MyModel
  self.table_name = 'my_table_name' #First table name, then tennat
  multi_tenant :account
end

class OtherModel
  has_one :some_model,
          class_name: 'MyModel'
end

#when executing
obj = OtherModel.find(...)
obj.some_model

• Produced query contains account_id

SELECT "my_table_name".* FROM "my_table_name" WHERE "my_table_name"."account_id" = 20 AND "my_table_name"."some_other_prop" = 10 LIMIT 1

This only happens if puma server is started with code like in example 1.

If on start of puma server the current code is as in example 2. - then account_id is always included in query, no matter of the order of the table_name and multi_tenat lines

Documentation

Where should one contribute documentation to? I mean - directly to the readme or is there some specific place for the documentation?

ActiveRecord::ConnectionAdapters::Column no longer supports 'type_cast_for_database'

Hello!

When upgrading a Rails project from rails 4 to rails 5 I noticed this functionality no longer works. When I investigated further I found that this method type_cast_for_database no longer exists for ActiveRecord::ConnectionAdapters::Column.

The rails change that made this break: rails/rails@b93b39e#diff-de807ece2205a84c0e3de66b0e5ab831325d567893b8b88ce0d6e9d498f923d1

Where this is leveraged:

activerecord-multi-tenant/lib/activerecord-multi-tenant/copy_from_client.rb

Line 12 in 5bc94d3

row = row.map.with_index { |val, idx| @column_types[idx].type_cast_for_database(val) }

after_initialize issue with partial arel `select`

a ActiveModel::MissingAttributeError is raised in the model_extensions.rb after_initialize block if the partition_key is not included in the custom select

was

        # New instances should have the tenant set
        after_initialize Proc.new { |record|
          if MultiTenant.current_tenant_id && record.public_send(partition_key.to_sym).nil?
            record.public_send("#{partition_key}=".to_sym, MultiTenant.current_tenant_id)
          end
        }

This can be fixed by:

1. editing app code to include partition_key in the select
2. checking that the partition_key is loaded before the public_send, eg

        # New instances should have the tenant set
        after_initialize Proc.new { |record|
          if MultiTenant.current_tenant_id &&
            (!record.attribute_present?(partition_key) || record.public_send(partition_key.to_sym).nil?)
            record.public_send("#{partition_key}=".to_sym, MultiTenant.current_tenant_id)
          end
        }

(this ought to work for cases of partition_key-not-being-a-column, since attribute_present? returns false for nonsense values, circa Rails 4.2.7.1)
3) add a method that rescues ActiveModel::MissingAttributeError while trying the public_send (in case there's some unmanageable override/alias preventing an attribute_present? check.

Do you want a PR for option 2?

Multi-tenant class not found if defined inside a module

The following code works as expected and the call to multi_tenant :tenant automatically calls belongs_to :tenant and defines both tenant and tenant= methods.

class Tenant < ApplicationRecord; end

class Foo < ApplicationRecord
  multi_tenant :tenant
end

However, if both classes are inside a module

module MyModule
  class Tenant < ApplicationRecord; end

  class Foo < ApplicationRecord
    multi_tenant :tenant
  end
end

the multi_tenant :tenant call does not automatically call belongs_to :tenant or define tenant and tenant= methods.
This is due to the implementation of tenant_klass_defined?

def self.tenant_klass_defined?(tenant_name)
  !!tenant_name.to_s.classify.safe_constantize
end

so only the top-level constant ::Tenant is expected to be the tenant class for tenant name :tenant, ::MyModule::Tenant is not considered.

I thought about copying Rails' implementation of compute_type

https://github.com/rails/rails/blob/6ef39975d60cc9dafd1728c49e394dad11d12327/activerecord/lib/active_record/inheritance.rb#L224-L235

that in our example would search for the following 3 classes:

::MyModule::Foo::Tenant, ::MyModule::Tenant, ::Tenant

in that order.

Another solution would be to pass class_name just like we do in Rails.

multi_tenant :tenant, class_name: "::MyModule::Tenant"

What do you guys think?

I'm willing to implement this if we can agree on a solution.

Hide "Active Record does not support composite primary key." warnings in Rails 5.0

When running the test suite under Rails 5.0+, or adding multi_tenant to a model, the following warning will show:

WARNING: Active Record does not support composite primary key.

projects has composite primary key. Composite primary key is ignored.

This is irrelevant, since we don't need Rails to support composite primary keys (in a way, this library adds it). It would be reasonable to hide this warning for tables that have multi_tenant defined.

¿Migrate or merge Tenants?

Is there a a way to migrate data from one Tenant to another?

It's to crazy what I'm asking?

MultiTenant Cache Issue

After upgrade from Rails 4.2 to 5.2, I am facing issue with multi tenant cache issue.

https://discuss.rubyonrails.org/t/apartment-multi-tenant-gem-find-method-issue/81803/6

Whenever I tried to query with Tenant 2 - It is getting the result with previous tenant value on find method.

Using tenant in inserts?

I can't get the tenant to be added to inserted records, is this expected behavior? If so, do you have plans to support this?

`Model.limit(n).delete_all` & `Model.limit(n).update_all` generates incorrect query

Sharing the queries so it is clear what is happening

MultiTenant.with(Account.find(1)) { Project.limit(1).delete_all } generates a SQL query

DELETE FROM "projects" WHERE "projects"."id" IN (SELECT "projects"."id" FROM "projects" WHERE "projects"."account_id" = 1 AND "projects"."account_id" = 1 LIMIT 1)

As you can see subquery has account_id condition added correctly but once it returns ids, top level query doesnt have account_id condition.

Ideally it should have generated

DELETE FROM "projects" WHERE "projects"."id" IN (SELECT "projects"."id" FROM "projects" WHERE "projects"."account_id" = 1 LIMIT 1) AND "projects"."account_id" = 1

Environment
Rails 6.1
Ruby 3.1
Citus 10.2

Accessing the association loaded status causes the association to actually load

This is a bit of a strange one for Rails 4.2. It looks fixed in 5.2 so I understand if the support isn't provided for it.

AR MT checks if the association is loaded (https://github.com/citusdata/activerecord-multi-tenant/blob/master/lib/activerecord-multi-tenant/model_extensions.rb#L74) and a few other items before returning the current_tenant. This should pretty much always be false (loaded?) if the object wasn't loaded via a preloaded query.

When Rails saves a record, it checks all belongs_to associations in memory to determine if they are needing to be saved as well. This causes the association of the tenant to be loaded from the DB. In Rails 4.2 this code is at https://github.com/rails/rails/blob/4-2-stable/activerecord/lib/active_record/autosave_association.rb#L431

Rails 5.2 fixes this because it only proceeds if the association is actually loaded https://github.com/rails/rails/blob/5-2-stable/activerecord/lib/active_record/autosave_association.rb#L469

Not able to load Gem in Rails 5.0 or Rails 5.1

I am having trouble configuring this gem to work in Rails 5.0 or 5.1.

basic tests are:

1. install Rails 5.x
2. create new Rails project rails new test_proj
3. update gemfile to include gem 'activerecord-multi-tenant', '0.5.0'
4. rails c (also get the same error with bundle exec rails c
5. The following error is generated:

/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:94:in `rescue in block (2 levels) in require': There was an error while trying to load the gem 'activerecord-multi-tenant'.
Gem Load Error is: uninitialized constant ActiveRecord::Relation::QueryMethods
Backtrace for gem load error is:
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation.rb:18:in `<class:Relation>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation.rb:5:in `<module:ActiveRecord>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation.rb:3:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation/from_clause.rb:2:in `<module:ActiveRecord>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation/from_clause.rb:1:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.3/lib/active_record/relation/query_methods.rb:1:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-multi-tenant-0.5.0/lib/activerecord-multi-tenant/query_rewriter.rb:32:in `<module:ActiveRecord>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-multi-tenant-0.5.0/lib/activerecord-multi-tenant/query_rewriter.rb:31:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-multi-tenant-0.5.0/lib/activerecord-multi-tenant.rb:9:in `require_relative'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-multi-tenant-0.5.0/lib/activerecord-multi-tenant.rb:9:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `each'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `block in require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `each'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:107:in `require'
/home/vagrant/test_multi/config/application.rb:7:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:82:in `require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:82:in `preload'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:143:in `serve'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:131:in `block in run'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:125:in `loop'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application.rb:125:in `run'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/spring-2.0.1/lib/spring/application/boot.rb:19:in `<top (required)>'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/home/vagrant/.rbenv/versions/2.4.1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
-e:1:in `<main>'

I'm probably just missing a step, but not clear what is is just yet.

switch tenants for existing records

Hi, wonderfull gem, but I'm not being able to find more doc information, is there a way to switch the associated tenant for an specific record?, I'm always getting MultiTenant::TenantIsImmutable: MultiTenant::TenantIsImmutable.

Thanks for the help.

Deprecation warning Rails 6.0?

After installing this gem on a Rails 6 application I get a deprecation warning when running my rspec test suite:

DEPRECATION WARNING: Initialization autoloaded the constants ActionText::ContentHelper and ActionText::TagHelper.

Being able to do this is deprecated. Autoloading during initialization is going
to be an error condition in future versions of Rails.

Reloading does not reboot the application, and therefore code executed during
initialization does not run again. So, if you reload ActionText::ContentHelper, for example,
the expected changes won't be reflected in that stale Module object.

These autoloaded constants have been unloaded.

Please, check the "Autoloading and Reloading Constants" guide for solutions.

Not sure what this warning has to do with this gem but if I remove the gem it goes away. Any thoughts?

New release?

The current version v1.1.1 was released on 16 Jan 2021. v1.1.1 does not include support for Rails 6.1 or later #132 #126
Do you have any plan to release a new version?

Adding a tenant to a record persisted with tenant nil

I have a case where I persist a record of request input (email processing) before evaluating which tenant it should be assigned to.

When trying to add a tenant id later, persistence silently fails due to

activerecord-multi-tenant/lib/activerecord-multi-tenant/query_rewriter.rb

Lines 227 to 233 in 4512aee

	def update(arel, name = nil, binds = [])
	model = MultiTenant.multi_tenant_model_for_arel(arel)
	if model.present? && !MultiTenant.with_write_only_mode_enabled? && MultiTenant.current_tenant_id.present?
	arel.where(MultiTenant::TenantEnforcementClause.new(model.arel_table[model.partition_key]))
	end
	super(arel, name, binds)
	end

seeing MultiTenant.with_write_only_mode_enabled? false and a tenant (temporarily) set, so the update has a WHERE clause for the target id when the database record currently has nil.

The tenant is temporarily set by

activerecord-multi-tenant/lib/activerecord-multi-tenant/model_extensions.rb

Lines 96 to 110 in 4512aee

	around_save -> (record, block) {
	if persisted? && MultiTenant.current_tenant_id.nil?
	MultiTenant.with(record.public_send(partition_key)) { block.call }
	else
	block.call
	end
	}
	around_update -> (record, block) {
	if MultiTenant.current_tenant_id.nil?
	MultiTenant.with(record.public_send(partition_key)) { block.call }
	else
	block.call
	end
	}

I considered whether such code should generally use attribute_was like

        around_update -> (record, block) {
          if MultiTenant.current_tenant_id.nil?
            MultiTenant.with(record.attribute_was(partition_key)) { block.call }
          else
            block.call
          end
        }

But that might be vulnerable. perhaps it should only allow changing from untenanted nil to tenanted like

        around_update -> (record, block) {
          record_tenant = record.attribute_was(partition_key)
          if MultiTenant.current_tenant_id.nil? && !record_tenant.nil?
            MultiTenant.with(record_tenant) { block.call }
          else
            block.call
          end
        }

Or perhaps be further restricted to some setting on the model to allow nontenant=>tenant.
Perhaps this would be a useful pattern for upgrading a single model/table to tenanted, rather than setting an app-wide MultiTenant.enable_write_only_mode

Thoughts/Suggestions?