artur-sulej / excellent_migrations

An Elixir tool for checking safety of database migrations.

Hey there! My name is Artur and I'm a software engineer with a passion for DIY projects and anything creative. Programming is my way of bringing my ideas to life without making a mess in my house. In addition to coding, I'm an avid motorcyclist and coffee connoisseur.

I started my career as an Android developer before moving on to backend with Ruby and Elixir.

When I'm not working, you can find me spending quality time with my family as a happy husband and dad.

I also give talks – you can find them here:

• Safer DB migrations with excellent_migrations (upcoming) @ ElixirConf EU 20.04.2023
  • Event page
  • Repository
• Deumbrellization – cheap microservices @ warsaw.ex Meetup 23.02.2023
  • Talk video
  • Repository
  • Event page
• WebSocket in Elixir @ warsaw.ex Meetup 08.11.2018
  • Event page

Thanks for stopping by my profile and I hope you have a great day!

excellent_migrations's Issues

Architecture of detectors

It's a good idea to tackle complexity of finding unsafe operations – to make them easier to read, understand, maintain and develop.
There two aspects:

• there are many checks
• some checks are compound – they need to check conditions on multiple lines (e.g. index added concurrently and lock disabled)

Possible approach is to make AstParser return relevant parts of code (not yet detected unsafe operations). Then add another step to the process that would decide about dangers (based on that data from AstParser).
I like this approach, because it would decompose the logic into separate, independent steps with different responsibilities.

The question is: will this approach be suitable for all cases, including detecting irreversible migrations.

We've already had some discussion about it with @hiagomeels here.

Incorporate "Safe Ecto Migrations" Content Into Credo Checks Where Possible

Prominent Elixir dev @dbernheisel has talked about how to do safe Ecto migrations in a couple places online:

• Blog series: https://fly.io/phoenix-files/safe-ecto-migrations/
• Repo: https://github.com/fly-apps/safe-ecto-migrations

On a recent episode of the Thinking Elixir podcast, he talks again about safe migrations (the topic starts at 7:49 and ends at 18:46).

At 15:07 another host asks if David is planning on adapting his writings/recipes as credo checks. David says he probably won't and cites excellent_migrations as a good existing tool for this (that's how I ended up here 👋).

My question: As far as y'all can tell, do the credo checks in this repo cover most of the cases that David outlines in the above series/repo? I know some of what David suggests is hyper-specific to your own application, but it'd at least be nice if the "easy stuff" was covered in the credo checks!

Thanks!

Release a new version?

👋 Can you release a new version that includes the fix for proper exit codes? We've been using 0.1.6 and inadvertently letting unsafe migrations pass our build checks.

Thank you!

index_not_concurrently should verify that DDL transactions/migrations lock are disabled

Hello 👋

First, thanks for working on this library. It seems helpful to enforce good practices while you do data migrations. In particular, I'm interested in the index_not_concurrently check. On my quick review, it seems to detect the lack of the concurrently option in your index., but apparently, the check doesn't verify for:

@disable_ddl_transaction true
@disable_migration_lock true

Both module attributes are recommended when you add or drop an index concurrently, from the Ecto docs we have:

PostgreSQL supports adding/dropping indexes concurrently (see the docs). However, this feature does not work well with the transactions used by Ecto to guarantee integrity during migrations.

Therefore, to migrate indexes concurrently, you need to set both @disable_ddl_transaction and @disable_migration_lock to true:

Disabling DDL transactions removes the guarantee that all of the changes in the migration will happen at once. Disabling the migration lock removes the guarantee only a single node will run a given migration if multiple nodes are attempting to migrate at the same time.

Since running migrations outside a transaction and without locks can be dangerous, consider performing very few operations in migrations that add concurrent indexes. We recommend to run migrations with concurrent indexes in isolation and disable those features only temporarily.

Also, based on the last paragraph, the calls that should be allowed while you have disabled the DDL transactions and the migration lock are create/drop index(...., concurrently: true), but I don't know if that's feasible with this check.

Feature request: Provide database and version to reduce false positives

If we require/allow the user to provide the name of their database and it's version then we could skip raising warnings when we know that the warning is not valid for that database and version combination. For example in the description of the "Adding a column with a default value" check we have the note:

Note: This becomes safe in:

• Postgres 11+
• MySQL 8.0.12+
• MariaDB 10.3.2+

If we provided either :postgres, :mysql, or :maria_db along with the version as part of the configuration of excellent_migrations then we could skip raising these warnings in the cases when we know they're safe.

FunctionClauseError

** (FunctionClauseError) no function clause matching in ExcellentMigrations.MessageGenerator.build_message/1    
    
    The following arguments were given to ExcellentMigrations.MessageGenerator.build_message/1:
    
        # 1
        :column_reference_added
    
    Attempted function clauses (showing 10 out of 14):
    
        def build_message(%{type: type, path: path, line: line})
        def build_message(:raw_sql_executed)
        def build_message(:index_not_concurrently)
        def build_message(:many_columns_index)
        def build_message(:column_added_with_default)
        def build_message(:column_removed)
        def build_message(:table_dropped)
        def build_message(:table_renamed)
        def build_message(:column_renamed)
        def build_message(:not_null_added)
        ...
        (4 clauses not shown)
    
    (excellent_migrations 0.1.1) lib/message_generator.ex:4: ExcellentMigrations.MessageGenerator.build_message/1
    (excellent_migrations 0.1.1) lib/message_generator.ex:5: ExcellentMigrations.MessageGenerator.build_message/1
    (excellent_migrations 0.1.1) lib/mix/tasks/check_safety.ex:23: anonymous fn/1 in Mix.Tasks.ExcellentMigrations.CheckSafety.run/1
    (elixir 1.13.0-rc.0) lib/enum.ex:937: Enum."-each/2-lists^foreach/1-0-"/2
    (excellent_migrations 0.1.1) lib/mix/tasks/check_safety.ex:21: Mix.Tasks.ExcellentMigrations.CheckSafety.run/1
    (mix 1.13.0-rc.0) lib/mix/task.ex:397: anonymous fn/3 in Mix.Task.run_task/3
    (mix 1.13.0-rc.0) lib/mix/cli.ex:84: Mix.CLI.run_task/2

I'm actually not sure which of my migrations triggered this, but the whole lot of them are at: https://github.com/pmarreck/mpnetwork/tree/yolo/priv/repo/migrations

Danger detected incorrectly for repos that use `migration_lock: :pg_advisory_lock`

My repo is configured to use the pg advisory lock strategy for locking migrations across nodes, which does not use a database transaction, and when I assure ExcellentMigrations that it's ok to not use the migration lock, and I'm creating an index concurrently, it still complains about not disabling the ddl transaction even though I have.

I traced it to this case statement that has a catchall clause to add the DDL Transaction danger, which seems incorrect.

Here's an example setup:

config :my_app, MyApp.Repo,
  migration_lock: :pg_advisory_lock

defmodule MyApp.Repo.Migrations.MyMigration do
  use Ecto.Migration
  @disable_ddl_transaction true

  # excellent_migrations:safety-assured-for-this-file index_concurrently_without_disable_migration_lock

  def change do
    create_if_not_exists index("foo", [:bar], concurrently: true)
  end
end

my_app ❯ mix credo
Checking 2682 source files (this might take a while) ...

  Warnings - please take a look
┃ 
┃ [W] ↗ Index concurrently without disable ddl transaction
┃       priv/repo/migrations/20240125215410_my_migration.exs:6 #(MyApp.Repo.Migrations.MyMigration.change)

Safety of rolling back non-concurrent drops

excellent_migrations won't complain about things like these:

def change do
  drop_if_exists index(:my_table, [:my_column])
end

Even though rolling back such a migration will recreate the index non-concurrently.

Changing this to:

def change do
  drop_if_exists index(:my_table, [:my_column], concurrently: true)
end

Will recreate the index concurrently when rolling back.

Since dropping an index by itself will acquire an ACCESS EXCLUSIVE lock on the table, I'm thinking maybe we should enforce the dropping of indexes to be concurrent to err on the side of caution.

Curious what others think of this.

Check is incorrect for index

based on recent change in Ecto SQL 3.9.x , it seems that it s not working correctly anymore this check. They added an advisory pg lock. per https://hexdocs.pm/ecto_sql/Ecto.Migration.html#index/3-adding-dropping-indexes-concurrently

I tried creating a migration that has a config of advisory lock but the check complained incorrectly until I added the following

@disable_migration_lock true

to my migration. This is wrong because the point of the advisory lock is to avoid removing all completely the lock?

  use Ecto.Migration

  @disable_ddl_transaction true

  def change do
    create table(:mytable) do
      add(:some_field, :uuid, null: false)

      timestamps()
    end

    create(unique_index(:mytable, [:some_field], concurrently: true))
end

I get an a credo error with "Index concurrently without disable ddl transaction" 

so then I have to fix it by added @disable_migration_lock true but based on the way it s supposed to work, I should not have to turn it off at all

I would be happy to help but not sure how this works

Non-concurrent index detection doesn't work with pipelines

The following code will not be flagged when checked against excellent_migrations. This is because the AST parser expects a different structure: https://github.com/Artur-Sulej/excellent_migrations/blob/master/lib/ast_parser.ex#L40

defmodule ExcellentMigrations.CreateIndexPipeline do
  def up do
    :dumplings |> index([:dough]) |> create()
    :dumplings |> index([:dough]) |> create_if_not_exists()
  end

  def down do
    :dumplings |> index([:dough]) |> drop()
    :dumplings |> index([:dough]) |> drop_if_exists()
  end
end

Is there a reason why we can't just look at all calls of index and unique_index whether inside create or not?

running excellent_migrations breaks on this migration

Here's the migration it breaks on:

https://github.com/pmarreck/mpnetwork/blob/yolo/priv/repo/migrations/20171016152948_add_additional_indexes_to_attachments_table.exs

Here's the error I see:

bash>> mix excellent_migrations.check_safety

==> excellent_migrations
Compiling 9 files (.ex)
Generated excellent_migrations app

14:07:05.637 [error] Task #PID<0.478.0> started from #PID<0.93.0> terminating
** (Protocol.UndefinedError) protocol Enumerable not implemented for :primary of type Atom
    (elixir 1.13.0-rc.0) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir 1.13.0-rc.0) lib/enum.ex:155: Enumerable.count/1
    (elixir 1.13.0-rc.0) lib/enum.ex:656: Enum.count/1
    (excellent_migrations 0.1.0) lib/parser.ex:63: ExcellentMigrations.Parser.detect_many_columns_index/1
    (excellent_migrations 0.1.0) lib/parser.ex:21: ExcellentMigrations.Parser.detect_dangers/1
    (excellent_migrations 0.1.0) lib/parser.ex:12: anonymous fn/3 in ExcellentMigrations.Parser.traverse_ast/2
    (stdlib 3.16.1) lists.erl:1358: :lists.mapfoldl/3
    (elixir 1.13.0-rc.0) lib/macro.ex:448: Macro.do_traverse/4
Function: &:erlang.apply/2
    Args: [#Function<1.59565726/1 in ExcellentMigrations.Runner.check_migrations/1>, ["priv/repo/migrations/20171016152948_add_additional_indexes_to_attachments_table.exs"]]
** (EXIT from #PID<0.93.0>) an exception was raised:
    ** (Protocol.UndefinedError) protocol Enumerable not implemented for :primary of type Atom
        (elixir 1.13.0-rc.0) lib/enum.ex:1: Enumerable.impl_for!/1
        (elixir 1.13.0-rc.0) lib/enum.ex:155: Enumerable.count/1
        (elixir 1.13.0-rc.0) lib/enum.ex:656: Enum.count/1
        (excellent_migrations 0.1.0) lib/parser.ex:63: ExcellentMigrations.Parser.detect_many_columns_index/1
        (excellent_migrations 0.1.0) lib/parser.ex:21: ExcellentMigrations.Parser.detect_dangers/1
        (excellent_migrations 0.1.0) lib/parser.ex:12: anonymous fn/3 in ExcellentMigrations.Parser.traverse_ast/2
        (stdlib 3.16.1) lists.erl:1358: :lists.mapfoldl/3
        (elixir 1.13.0-rc.0) lib/macro.ex:448: Macro.do_traverse/4

Using latest elixir and erlang, and a recent phoenix (1.5).

This is an old-ish migration so it's entirely possible that even though it runs (it does), it might be syntactically out-of-date

Feature Request: Configurable migrations paths

Currently when called by credo it's not possible to specify the migration path because it's hard-coded in relevant_file?/2:

  def relevant_file?(path, start_after) do
    !String.starts_with?(path, ["deps/", "_build/"]) &&
      !String.contains?(path, ["/deps/", "/_build/"]) &&
      String.contains?(path, "migrations/") &&
      migration_timestamp(path) > start_after
  end

This causes an issue because I'm using data migrations (similarly to how it's spelled out here: https://fly.io/phoenix-files/backfilling-data/) that are stored in priv/data_migrations/ and I'd like excellent migrations to ignore them because I do plan to use Repo operations within them (and they have a whole host of other considerations).

Credo checks integration broken in 0.1.7

When I upgrade from 0.1.6 to 0.1.7 I no longer get excellent_migrations errors via Credo (I'm testing with Credo 1.7.7 but it appears to apply to most any version).

The culprit appears to be #24

The PR itself looks fine to my eyes so I wonder if there is another part of excellent_migrations that was inadvertently depending on the swap.

I also tested with 0.1.7 but undoing the swap and when I do that I do get excellent_migrations error reported by mix credo.

Add more example migrations & tests for them

Let's keep cooking theme in tables/columns names for consistency and fun 🧑‍🍳
I believe it's a great introductory task, if you want to contribute (but not the only one).

You can catch me on Elixir slack @Artur Sulej for a chat about details.

Non-zero status is not returned when safety checks fail

Problem

I noticed that despite the implementation, the Mix task does not appear to return a non-zero exit status when dangers are found. Here's an example:

» mix excellent_migrations.check_safety; echo $status

09:08:12.638 [warning] Index not concurrently in priv/repo/migrations/20230103220859_testing-stuff.exs:5
0

Note: Echo'd status is 0.

Solution

I'm not certain why this is an issue, but I'm guessing that perhaps System.stop(1) isn't behaving as expected with Mix? Perhaps there's another way to return non-zero status from Mix, such as raising an error after logging dangers, https://hexdocs.pm/mix/Mix.html#raise/2.

I love the library <3

Cheers

Avoid compilation warnings when migrating with safety_assured

Hi, thanks for the library! I'm currently integrating it into a project.

Is there a way to add a @safety_assured annotation without causing a compilation warning? Currently I get warnings like this:

$ mix ecto.migrate
warning: module attribute @safety_assured was set but never used
  priv/repo/migrations/20211105021748_not_null_fields.exs:3

Although to fix this, the approach for marking safety assured might need to be changed.

Maybe instead of:

@safety_assured [:not_null_added, :column_type_changed]

It could be written as:

ExcellentMigrations.safety_assured([:not_null_added, :column_type_changed])

Feature Request: Reversible migrations

It would be great if excellent_migrations could enforce that a migration is reversible as a check.