drfraker / snipe-migrations

Blazing fast database migrations for Laravel tests.

Snipe Migrations

Blazing fast database migrations for Laravel tests.

The package takes a snapshot of your mysql database and imports the schema to your test database rather than running all of your migrations when the test suite starts up.

If you have a project with many migration files, this process can provide you with a massive speed improvement when initializing your test suite. This package can be used as a replacement for the RefreshDatabase trait that is provided out of the box with Laravel.

As an example, we tested this on an application that takes about 4 seconds to run all migrations with RefreshDatabase. Using SnipeMigrations the tests start up in 200 ms.

Requirements

1. Laravel >= 5.5
2. PHP >= 7.1
3. MySql or MariaDb, with separate database for testing.
   • For example if you have a development database for your application called amazingapp you would create a test database called amazingapp_test and add the details of the database in your phpunit.xml file. amazingapp_test is the database that Snipe will keep in sync for you.

Installation

Require the package using composer.

composer require --dev drfraker/snipe-migrations

Usage

After you've installed the package via composer

1. Add the SnipeMigrations trait to your tests/TestCase file. Don't forget to import the class at the top of the file. Once you have added the SnipeMigrations trait, simply use the RefreshDatabase trait in any tests in which you wish to use database access, and Snipe will handle the rest for you.

When you're done, your tests/TestCase.php file should look like this.

<?php

namespace Tests;

use Drfraker\SnipeMigrations\SnipeMigrations;
use Illuminate\Foundation\Testing\TestCase as BaseTestCase;

abstract class TestCase extends BaseTestCase
{
    use CreatesApplication, SnipeMigrations;
}

2. By default, SnipeMigrations will store a .snipe file and the snipe_snapshot.sql file in /vendor/drfraker/snipe-migrations/snapshots. If you would like to change the location of the files follow the directions below to publish the snipe config file.

   • To publish the snipe config file, run php artisan vendor:publish and select snipe-config from the list.

3. If the snipe_snapshot.sql file gets out of sync for any reason you can clear it by running php artisan snipe:clear. After running this command your original database migrations will run again creating a new snipe_snapshot.sql file.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

snipe-migrations's Issues

Add Tests

The package needs more tests before adding any additional features.

Silently fails with exec() when mysql is not installed

I neither have mysql nor mysqldump running on my system. We use docker (and docker-compose) to run our services and thus don't need them installed locally.

Since snipe-migrations uses exec() in the background, this silently fails in case these binaries are not installed.

For me, the fix would be to call docker-compose mysql_test {command} (e.g. docker-compose mysql_test mysqldump ...).

It would be great if there could be a way to either:

• specify a prefix (or binary location) or
• call the binary "by hand"

using methods that can be overwritten by the actual test case like so (pseudo-code):

class TestCase {
  use SnipeMigrations;

  protected function getSnipeBinaryPath()
  {
    return 'docker-compose mysql_test`;
  }
}

(or another, suitable method name).

"PHP >= 7.1" but composer rejects 8

The documentation says "PHP >= 7.1" but composer.json rejects 8 with
"php": "^7.1"

Consider changing composer.json to say
php": "^7.1|^8.0",

or, more precisely in line with the documentation,
"php": ">=7.1"

Running tests results in "The "" directory does not exist."

I installed Snipe-Migrations according to the Readme, but running tests that use RefreshDatabase, the package is unable to create a snapshot and populate the database.
Instead, I get an error telling me that "The "" directory does not exist.".

I did some digging and a possible issue might be that "app()['migrator']->paths()" includes an empty entry at index 0. Not sure if that's normal or where that's coming from. At least that's the point where the script seems to fail.

Show progress

Currently there is no way to see the progress of the migration process before a test.

Would you be opposed me adding a feature that shows the migration / seeding progress. This could be (default) hidden behind a config setting. Let me know if this is something that could be merged and I will implement it.

Does not work with remote database host

It lacks -h hostname clause for mysqldump and mysql commands.

Best way to integrate it with CI process ?

How would you approach this? I was thinking of

1. adding the .snipe and snipe_snapshot.sql to the version control
2. a pre commit hook that would run the test locally and commit the .snipe and snipe_snapshot.sql together so that the first run on the github CI server wouldn't be necessary ?

PHP version `^7.1` or `^7.2`?

Readme says that package requires PHP >= 7.1, but the composer.json wants ^7.2.

Do we really need ^7.2? 🙂

Is this package still maintained?

I see several tickets and PR's that are over a year old. Do you need help with maintaining? Or does this package not benefit test performance in newer Laravel versions anymore?

Configuration example

Hi Dustin,

I am very much interested in giving your package a try-out, the application I maintain has a very large test-suite that it seems would benefit greatly from this approach.

I am hoping if you would kindly consider adding an example of configuration for the databases ( .env, .env.testing, phpunit.xml)

, I am not able to get the test database to work, and it is unclear if this DB needs to be migrated when tests are exectuded the first time. I have created the database "projectName_test" but my tests cannot connect to it, I tried all combinations it seems.

Thanks

SQLite support

I know you've mentioned SQLite support in the future but was just wondering if you have any thoughts on the implementation.

For starters, would it be SQLite to SQLite setup or would it allow MySQL to SQLite setup? We use MySQL in production but SQLite in our tests.

We already have something similar to this package that we use to facilitate the MySQL to SQLite migration for testing, but implementing it in a package like this looks a little tricky because I can't see an easy way to get the database credentials for each database as when the tests are running, you cant get the MySQL credentials as they are normally stored in environment vars.

Have you any thoughts on this?

Windows-specific issues?

When I was setting this up on my Windows (localhost), I ran into a couple of problems:

1. I had to add MySQL to my PATH so that the mysqldump command would work. Maybe this should be stated on the README? It wasn't immediately obvious to me, at least.

2. The "2>/dev/null" at the end of the mysqldump call gives the output "The system cannot find the file specified" and results in an empty snipe_snapshot.sql file. Removing it fixes the error and the sql file is then populated.

Add postgres support

is it planned?

False compatibility statement with laravel version 5.2

Does not work for me with 5.4.
Relies on migrate:fresh command to execute, but that one has been introduced in laravel 5.5.

There is no default value for config('snipe.snipefile-location')

It seems like there is no default configuration and file_put_contents does not seem to like an empty file name. I haven't published the configurations yet. It might fix it, but just letting know that it does not work out-of-the-box.

file_put_contents(config('snipe.snipefile-location'), $timeSum);

ErrorException: file_put_contents(): Filename cannot be empty
   │ 
   │ /var/www/pillar.science/src/api/vendor/drfraker/snipe-migrations/src/Snipe.php:49
   │ /var/www/pillar.science/src/api/vendor/drfraker/snipe-migrations/src/Snipe.php:20
   │ /var/www/pillar.science/src/api/vendor/drfraker/snipe-migrations/src/SnipeMigrations.php:15
   │ /var/www/pillar.science/src/api/vendor/laravel/framework/src/Illuminate/Foundation/Testing/TestCase.php:71
   │ /var/www/pillar.science/src/api/tests/Functional/Services/Search/SearchServiceTest.php:44

I guess just adding a default value to config('snipe.snipefile-location', 'default-location.txt') might do the trick here.

Is it possible to support Snipe Migrations on the DatabaseMigrations trait?

I'm using the DatabaseMigrations trait, instead of the RefreshDatabase trait, as I'm running my tests through Dusk.

Is it possible for Snipe Migrations to work with DatabaseMigrations as well?

Cannot use non default mysql port

It seems that the mysqldump execute doesn't include the port parameter, so using a non default port is not supported.