dania02525 / apartmentex

SaaS support for Phoenix + Ecto

Apartmentex

Easy SaaS for Phoenix/Ecto.

In this branch, the following versions are supported:

• Ecto 2.0.x
• Postgrex 0.11.0

Features

• Tenant-qualified queries targeting postgres schemas or MySql databases
• Automatic migrations for tenant tables to schema for Postgres or database for MySQL

See an example app at https://github.com/Dania02525/widget_saas

Setup

• Add this to your mix.exs deps:

{:apartmentex, "~> 0.2.3"}

• Run: mix deps.get && mix deps.compile

• You can also configure your tenant schema prefix, adding this to your application configs:

config :apartmentex, schema_prefix: "prefix_" # the default prefix is "tenant_"

Use

Generate a migration file that will be run against each tenant:

mix apartmentex.gen.migration CreateUsers

By default, the migration will be generated to the "priv/YOUR_REPO/tenant_migrations" directory of the current application but it can be configured to be any subdirectory of priv by specifying the :priv key under the repository configuration.

You can now add a new tenant and automatically create a new schema for Postgres users or a new database for MySQL users, and run the migrations in priv/YOUR_REPO/tenant_migrations for that schema or database.

Table references and indexes in a migration will be applied to the same tenant prefix as the table within tenant_migrations.

Apartmentex.new_tenant(Repo, tenant)

When you need to update a tenant's schema based on new migrations, you can run:

# Runs all migrations necessary for the tenant, based on that tenant's
`schema_migrations` table
# Returns a tuple that is either:
# {:ok, prefix_of_tenant, versions_migrated}
# {:error, prefix_of_tenant, db_error_message}

{status, prefix, versions_or_error} = Apartmentex.migrate_tenant(Repo, tenant)

If there is a problem with a migration, you can roll it back by passing in the version (as an integer). This will revert every migration back until the version specified (including that version):

# Returns a tuple that is either:
# {:ok, prefix_of_tenant, versions_rolled_back}
# {:error, prefix_of_tenant, db_error_message}

{status, prefix, versions_or_error} = Apartmentex.migrate_tenant(Repo, tenant, :down, to: 20160711125401)

When deleting a tenant, you can also automatically drop their associated schema or database (for MySQL).

Apartmentex.drop_tenant(Repo, tenant)

Include the tenant struct or tenant id in Apartmentex calls for queries, inserts, updates, and deletes.

  widgets = Apartmentex.all(Repo, Widget, tenant)

  Apartmentex.insert(Repo, changeset, tenant)

  Apartmentex.update(Repo, changeset, tenant_id)

  Apartmentex.delete!(Repo, widget, tenant_id)

To Do

• mix task to migrate or rollback all tenant schemas/databases

apartmentex's Issues

Support for UUIDs

I use UUIDs for all my primary keys on my tables.
However when trying to use apartmentex it calls out to the erlang function :erlang.integer_to_binary("87d59c0b-8182-4442-9f57-a44104aac81f") which does not work as a UUID is not a integer.

Thanks 😄

Migrating existing tables to multi tenant

Hi there!

I already have a phoenix app with a bunch of tables, it actually works in single tenant mode. I'd like to switch some of these tables to multi tenancy.

My clues on this is the following workflow:

• backup entire database
• move/edit migrations to priv/MY_REPO/tenant_migrations
• reset database
• run migrations from scratch
• re-import database data into a specific tenant

I'm just wondering if there's a better/safer/quickier way to do this. Have you got any advice, documentation or book to suggest?

Thanks in advance!

Foreign key issue

Hi. I have one question. If I create a table that have a foreign key references to tenant's table (fk_id_tenant). How can I create this relation?

Fix "redefining module" warnings in test suite

I noticed that there are a bunch of warnings when running the test suite like this:

warning: redefining module Apartmentex.TestPostgresRepo.Migrations.CreateTenantUser (current version defined in memory)
  priv/repo/tenant_migrations/20160711125401_test_create_tenant_notes.exs:1

I think this has to do with the way the Ecto.Migrator is loading the code in the migration files, but I'm not sure if the fix should be within Ecto or within Apartmentex. Thoughts?

Here's where the migrator loads the code in the migration files:
https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/migrator.ex#L239

New Patch Release?

I would like to use Apartmentex in an application but it looks like the current version posted to hex does not support the most recent versions of Ecto and Postgrex. However, it looks like there are commits in master that support the newer versions of these libs. Is there anything holding back a new patch release? If so, I would be glad to help out since I would rather not have to reinvent the wheel on my side.

Dump a Sample Tenant Structure

I've been using Apartmentex for a bit and I've been loving how it's practically just a simple wrapper around Repo + sugar.

Something that I'm interested in is a DB dump of tenant migrations. This way when you create a new tenant, instead of running migration after migration, you can instead load up some SQL and get the signup process finished earlier.

Perhaps something like a mix ecto.apartment_dump. I could set up a dummy apartment tenant with no prefix (this way it doesn't conflict in case an "apartment" tenant signs up). Then treat it as a normal tenant when you run migrations. Post migrations, a DB dump could get run for just that search path on development to be committed.

Thoughts?

Unable to run tenant_migrations on production remote_console

Hi there at the moment am unable to create a new tenant (run the migrations on the new tenant) on a production deployment. It is complaining about Mix which isn't normally present in production - any ideas what I am doing wrong - am using Edeliver/distillery to do the release/deploy
Thanks

Error on the remote_console on the prod instance

iex([email protected])5> Apartmentex.new_tenant(Repo, foo.id)
** (UndefinedFunctionError) function Mix.Project.deps_paths/0 is undefined (module Mix.Project is not available)
Mix.Project.deps_paths()
(ecto) lib/mix/ecto.ex:172: Mix.Ecto.source_repo_priv/1
(apartmentex) lib/apartmentex/migrations_path_builder.ex:6: Apartmentex.MigrationsPathBuilder.tenant_migrations_path/1
(apartmentex) lib/apartmentex/tenant_actions.ex:61: anonymous fn/4 in Apartmentex.TenantActions.migrate_and_return_status/4
(apartmentex) lib/apartmentex/tenant_actions.ex:72: Apartmentex.TenantActions.handle_database_exceptions/1
(apartmentex) lib/apartmentex/tenant_actions.ex:57: Apartmentex.TenantActions.migrate_and_return_status/4

function Mariaex.Error.message/1 is undefined

While tinkering with Apartmentex, I received a warning letting me know that function Mariaex.Error.message/1 is undefined.

$ mix do deps.get, deps.compile
* Updating apartmentex (https://github.com/BenMorganIO/apartmentex.git)
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
From https://github.com/BenMorganIO/apartmentex
 * [new branch]      postgrex-bump -> origin/postgrex-bump
Running dependency resolution
==> fs (compile)
==> ranch (compile)
==> poolboy (compile)
cc -g -O3 -Wall -I/usr/local/Cellar/erlang/18.2.1/lib/erlang/erts-7.2.1/include -Ic_src -fPIC -shared -dynamiclib -undefined dynamic_lookup -o priv/bcrypt_nif.so c_src/bcrypt_nif.c c_src/blowfish.c
==> cowlib (compile)
==> cowboy (compile)
==> apartmentex
Compiling 5 files (.ex)
warning: function Mariaex.Error.message/1 is undefined (module Mariaex.Error is not available)
  lib/apartmentex/tenant_actions.ex:73

Generated apartmentex app

Customise migration folder

Currently you can use the priv option on the repo to change the path used for the migrations e.g.

priv: "mydir"

will mean the migrations are stored in

priv/mydir/tenant_migrations.

I'm building an API where the tenants are referred to as services and I'd like the migrations directory to line up with that e.g.

priv/repo/service_migrations

How would you feel about making the priv option specify the whole path instead of just the part before tenant_migrations?

e.g.

priv: "repo/service_migrations"

means the migrations are stored in

priv/repo/service_migrations

pool size?

I was asking about pool_size setting in config here:
https://elixirforum.com/t/ecto-pool-size/4120, Do you have any recommendation?

Thanks

Compilation fails: cannot import Mix.Ecto.build_repo_priv/1

$ mix deps.compile apartmentex
==> apartmentex
Compiling 5 files (.ex)

== Compilation error in file lib/apartmentex/migrations_path_builder.ex ==
** (CompileError) lib/apartmentex/migrations_path_builder.ex:2: cannot import Mix.Ecto.build_repo_priv/1 because it is undefined or private
    (elixir) src/elixir_import.erl:77: :elixir_import.calculate/6
    (elixir) src/elixir_import.erl:18: :elixir_import.import/4
could not compile dependency :apartmentex, "mix compile" failed. You can recompile this dependency with "mix deps.compile apartmentex", update it with "mix deps.update apartmentex" or clean it with "mix deps.clean apartmentex"

Elixir version: 1.6.0

The one question - is this ready for production?

I really didn't want to open an issue for this, but I couldn't find any other way to talk to you. 😶

Ecto 2.1 compatibility issues

It seems like the last version of Ecto and Apartmentex have some issues.
On my application migrations don't work anymore.
I tried to run all the library tests with Ecto 2.1, there some broken tests but it's related to the errors messages returns by Postgresql.

Did you try to use it met some problems when running the library with Ecto 2.1 in a real project?

Tenant schema as subdomain

I checked out the widget_saas project for sample of using apartmentex. Is there a way to have the tenant schema as the subdomain? Something like this: https://django-tenant-schemas.readthedocs.io/en/latest/ i.e. http://first-tenant.example.com instead of http://example.com/tenant/:first_tenant_id.

Feature: Flag a migration as tenant based.

When importing migrations from a package, it would be great if I could be able to flag a migration as supposing to run for tenants, rather than for the public schema.

This way, when I'm developing a package, I can Add a flag as a configuration to my package. That way when a user goes to add the latest migrations from a version update, they don't have to worry about C&P'ing them into the tenant_migrations folder.

How to use prefix for schema's name in the project

Hello,

I'm make some tests with the apartmentex and I have a doubt.

I'm try to use a prefix configuration for a schema name e for one reason do not works very well, always the application use the default prefix "tenant_".

My project is a umbrella project and inside a config.exs of the projects that's have a Ecto dependence I put:

config :apartmentex, schema_prefix: "client_"

Unfortanely do not work. I did a search and I found out that the

https://groups.google.com/forum/#!msg/elixir-lang-talk/5yX9hBNEXe0/7UYWeTKmjaYJ

Using the strategy of the previous post the configuration works:
config :tenant_prefix, :apartmentex, %{ schema_prefix: "client_" }

However I need change the call in apartmentex for:
Application.get_env(:tenant_prefix, :apartmentex).schema_prefix

A another way is change the "key" :apartmentex for the my application name. Ex:

config :my_app, schema_prefix: "client_"

And change the call in apartmentex for
Application.get_env(:my_app, :schema_prefix)

How should I proceed?

Sorry for english mistakes.

PS: I'm using Ecto 3.0. When the compatibility will be available in Hex?

How to run new migrations against tenant schemas?

Here is the situation:

• Tenants sign up: Apartmentex.new_tenant(Repo, tenant)
• Developer adds new migration file in priv/repo/tenant_migrations/
• Need to run migrations for all tenants

Maybe it makes sense to have an Apartmentex.migrate_tenantsI(Repo, tenants) function that would migrate each tenant forward.

Or maybe this should be a mix task?
mix apartmentex.migrate TenantModel
Note the TenantModel parameter, since apartmentex would have to query for all tenants to calculate the schema prefixes that it needs to migrate. Or I suppose maybe apartmentex could query the database for a list of all schema that start with the prefix?

How to deal with failure? What if a migration fails for one tenant and not others?

Sorry - lots of thinking out loud here 😄

Move Migrations from Scripts to Compiled Code

Hey there,

I've been going through trying to run Apartmentex.new_tenant/4 as async as possible. The main problem with this is that the same can be loaded twice at the same time and cause Code.load_file/2 which Ecto uses to raise an error.

To resolve this, I believe we should move the tenant migrations out of the priv directory. They need to be used dynamically from the codebase when a new tenant is created, which means on a production app, the files need to be reloaded quite frequently. This is a bit pointless. Just compile and store the modules in memory. This would also enable us to use more dynamic column naming and indexes; although I'm not sure why this is useful or what use cases this supports, but yeah, that would become possible.

Anywho, I'm thinking the user can define where they would like their migrations to be stored. For me, I would place them a context relevant to the model that manages the tenants. Say each App.Accounts.User has a user_#{id} tenant. Then I would store them in: lib/app/accounts/user_migrations.

What are some thoughts on this? I have read a lot of Ecto migration code lately to dissect this problemo. I wouldn't mind being the person to build this out.

Does this work with Ecto 2.0.0-beta.1?

Prefix in `Apartmentex.drop_tenant` not being escaped properly?

I'm trying to get apartmentex working for me. I'm assuming I'm not doing anything too crazy :)

I'm using "slugs" as the tenant identifier on my Postgres schemas (I'm not using integer IDs, so would end up using a UUID, or a slug). So for example, my database schemas look like tenant_a-test-person.

Adding a tenant is working just fine. For example:

Apartmentex.new_tenant(Repo, client.slug)
# --> Apartmentex.new_tenant(Repo, "a-test-person")

However, when I go to drop the schema, I'm getting errors due to the presence of - in the schema name. For example:

Apartmentex.drop_tenant(Repo, client.slug)
# --> Apartmentex.drop_tenant(Repo, "a-test-person")

This ends up calling DROP SCHEMA tenant_test-person CASCADE [] in the DB, and throws an error, as per below:

[debug] QUERY ERROR db=0.5ms
DROP SCHEMA tenant_test-person CASCADE []
[debug] QUERY OK db=0.2ms
rollback []
{:error, :drop_schema,
 %Postgrex.Error{connection_id: 70573, message: nil,
  postgres: %{code: :syntax_error, file: "scan.l", line: "1053",
    message: "syntax error at or near \"-\"", pg_code: "42601", position: "24",
    routine: "scanner_yyerror", severity: "ERROR"}}, %{}}

I think all that needs to happen here is surround the schema name in quotes, e.g. DROP SCHEMA "tenant_test-person" CASCADE [] - at least it worked for me testing in a PG shell.

Is there a way to accomplish this with the library as-is? I don't think using UUIDs (instead of a slug) would make any difference here, as they also include - characters.

Thanks for your help and putting this library together!

** (Postgrex.Error) ERROR (undefined_table): relation "locations" does not exist

Hi there,

I'm trying to define my first "model" for a table that exists within each tenant's PG schema (i.e. it is created via a migration in priv/repo/tenant_migrations.

I define the Ecto schema as per normal:

  schema "locations" do
    field :address
    # ......

    timestamps()
  end

But it appears that Ecto can't find the table:

     ** (Postgrex.Error) ERROR (undefined_table): relation "locations" does not exist
     stacktrace:
       (ecto) lib/ecto/adapters/sql.ex:463: Ecto.Adapters.SQL.struct/6
       (ecto) lib/ecto/repo/schema.ex:397: Ecto.Repo.Schema.apply/4
       (ecto) lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
       (ecto) lib/ecto/repo/schema.ex:124: Ecto.Repo.Schema.insert!/4
       test/myapp/locations/location_test.exs:24: (test)

The locations table does not exist in my public schema, but it does inside each tenant's schema.

Am I missing something?

I am using ecto 2.X, postgrex ~> 0.12.1 and apartmentex ~> 0.2.2.

Having trouble with install

Any thoughts on what I am missing?

https://github.com/Dania02525/apartmentex/blob/master/lib/apartmentex.ex#L6

Broken deps as Ecto was upgraded

Hi,

As Ecto was updated the module Apartmentex.MigrationsPathBuilder no longer works since build_repo_priv/1 was removed/rename/made private from the project:

defmodule Apartmentex.MigrationsPathBuilder do
  import Mix.Ecto, only: [build_repo_priv: 1]
  @migrations_folder Application.get_env(:apartmentex, :migrations_folder) || "tenant_migrations"

  def tenant_migrations_path(repo) do
    Path.join(build_repo_priv(repo), @migrations_folder)
  end
end

Any clue of the new method/fix for this?

Reference at:
elixir-ecto/ecto#2278