mixd / packerwp

WordPress deployment written in PHP

PackerWP

Built by Mixd

PackerWP is a custom Deployer runbook. It was designed as the successor to the legacy Ruby deployment tool WP Deploy and has been tailored to include all the Capistrano tasks that were previously provided by WP Deploy.

Available tasks

Task	Description
backup-local-db	Backup a copy of a local database and upload it to a remote host
backup-remote-db	Backup a copy of a remote database and download it
deploy	Deploy your project
rollback	Rollback to previous release
pull	Pull both WordPress uploads and a database from a given host
pull-remote-db	Pull down a copy of the database from the remote host and import it into your local env
pull-remote-uploads	Pull media from a remote host
push-local-db	Push up a local copy of a database and import it into the remote host
push-local-uploads	Push media to a remote host
reset-admin-pwd	Reset the super admin password on the target environment
setup-local-wp	Set up your project locally
setup-wp	Set up your project on the remote host
ssh	Connect to host through ssh
debug:task	Display the task-tree for a given task
deploy:unlock	Unlock broken deployment

This tool has been designed around Mixd's NHS Framework though should be portable enough to drop into any other WordPress-based project.

Starting a new project

To get started, you'll need to install PackerWP into your project using composer

$ composer require mixd/packerwp

With PackerWP now installed, you should make sure you have the Deployer binary installed to your local system to make it easier to run commands.

$ curl -LO https://deployer.org/deployer.phar
$ mv deployer.phar /usr/local/bin/dep
$ chmod +x /usr/local/bin/dep

Setting up

PackerWP uses a deploy.json file in the root of your project to define all of the environment config.

Run cp ./vendor/mixd/packerwp/config/deploy.example.json ./deploy.json to get a copy of the accepted config rules added to the root of your project.

Go ahead and start populating it.

Now, add a symbolic link to the root of the project to link up the custom Deployer tasks. ln -s vendor/mixd/packerwp/deploy.php.

You should now be able to run dep and see a list of all the available tasks.

Tasks

Initial WordPress Installation

Once you have your environment set up, you can install WordPress by running

$ dep setup-local-wp

This task will generate a wp-config.php for you and place it in the project root, then it will run the WordPress installation, reset the WordPress Salts and present you with the new admin password and a link to the WordPress Admin.

If you need to re-roll the admin password, you can reset it at anytime by running

$ dep reset-admin-pwd [stage]

if you omit [stage] you will reset the password for the local environment, otherwise you can supply staging or production appropriately.

Note: This task will also re-roll the WordPress salts.

Working with databases

Occasionally you may wish to push or pull your MySQL Database to a specific environment.

Ensure that you have populated the wp_home_url and db_$ entries in your deploy.json for your target environment.

To download a copy of the database from a remote host and import it into your local environment you can run:

$ dep pull-remote-db [stage]

To upload your local database and import it into the remote host you can run:

$ dep push-local-db [stage]

Both the pull and push tasks will handle searching and replacing the necessary URLs to convert from local to [stage] or from [stage] to local.

If you wish to simply make a backup without importing or replacements, you can run:

$ dep backup-remote-db

or

$ dep backup-local-db

each of these tasks will make a MySQL Backup using wp db export and gzip it into a db_backups folder in the project root.

Working with uploaded media

To download the WordPress upload folder from a remote host run:

$ dep pull-remote-uploads [stage]

To upload your local WordPress upload folder to a remote host run:

$ dep push-local-uploads [stage]

Deployment

If you're ready to deploy your work to a remote host, simply run:

$ dep deploy [stage]

By default, PackerWP is configured to name every release using PHP's date function in the following format Ymd.

Templates

PackerWP also supports wp-config.php extras. If you want to override the WP Config extras that get added during the dep setup-wp task then do the following:

Create the templates folder: mkdir ./templates/

Copy the bundled WP Extras file: cp vendor/mixd/packerwp/config/templates/extras.php ./templates/extras.php

Now you can edit the extras file. Be sure to commit and push this file to Github so it persists for other developers.

Contributing

If you'd like to request changes or additional tasks we encourage you to create a Pull Request on the Github repo.

packerwp's Issues

upload/download tasks not applying correct flags

Currently the download and upload tasks are written as

download($src, $dest, ["flags" => "rzcE"])

which is invalid.

The correct syntax should be:

download($src, $dest, ["options" => ["flags" => "-rzc"]]);

Add task to auto-enable plugins after a setup completes

@roryashfordbentley:
It would be great if during the installation of a new project using the framework if we could enable certain key plugins. As an example ACF is required and without being enabled the site doesn't work (fatal error undefined function get_field()
https://github.com/Mixd/NHS-Framework/issues/138

Update docs to include all commands

Thre are a few features such as setup-wp that arent documented in the readme, it would be super helpful to see them all listed in a table perhaps

Shell command not working for me

The below command was not working on my server and I got a Exit Code: 2(Misuse of shell builtins), when I got rid of the first '' it started working.

PackerWP/deploy.php

Line 45 in d9980aa

$cmd = "sed -i '' 's" . $seperator . $before . $seperator . $after . $seperator . "g' \"$file\"";

Formatting and PSR-12

In keeping with our other projects we need to update the codebase to make it more readable, use docblock comments and do some PSR-12 linting potentially

IF-statement not functioning on my environment

The code below did not work properly on my set-up since curl was giving me a HTTP/2 200 instead of 200 OK. I switched the logic around and used strpos to fix my issue, see below.

Original:

PackerWP/config/tasks/heartbeat.php

Lines 17 to 31 in d9980aa

	if ($status_code !== "200 OK") {
	writeln("<error>Website did not return a 200 OK. Deployment assumed to have failed</error>");
	$confirm = askConfirmation("
	Do you wish to initiate an immediate rollback?",
	false
	);
	if ($confirm !== true) {
	invoke('rollback');
	invoke('deploy:unlock');
	exit;
	}
	} else {
	writeln("<info>Website responded with 200 OK. Safe to continue</info>");
	return;
	}

My version:

<?php 
    if (strpos($status_code, '200') !== false) {
      writeln("<info>Website responded with 200 OK. Safe to continue</info>");
      return;
    } else {
      writeln("<error>Website did not return a 200 OK. Deployment assumed to have failed</error>");
      $confirm = askConfirmation("
      Do you wish to initiate an immediate rollback?",
          false
      );
      if ($confirm == true) {
          invoke('rollback');
          invoke('deploy:unlock');
          exit;
      }
    }

Also the $confirm was somehow backwards?

DISALLOW_FILE_MODS should be set within wp-config for all environments.

DISALLOW_FILE_MODS is a constant that disables file editing within WordPress admin.

By setting the constant to true within wp-config.php it completely disables the backend file editor enhancing the security of the site and preventing accidental edits.

wp-config passwords not correctly escaped causing an early exit

If an & char is included in the db password, then during the wp setup task when the wp-config is written, the sed command does not complete as & is not escaped.

Extend range of ignored tables on backup db tasks

The dedicated searchwp tables are already being ignored for both remote and local backups.

Two other groups of tables could potentially be ignored, which would greatly reduce the size of the databases involved in push/pull operations on some sites: wsal_ (WP Activity Log tables) and swpext_ (Search WP extensions, such as metrics)

Both the 'backup-remote-db' and 'backup-local-db' tasks have a test for searchwp tables:
// Test for searchwp tables $cmd = '{{bin/wp}} db tables --all-tables "' . $prefix . 'searchwp*" --format=csv';

This could easily be extended to cover additional tables, e.g.:
// Test for searchwp and activity log tables $cmd = '{{bin/wpl}} db tables --all-tables "' . $prefix . 'searchwp*" "' . $prefix . 'wsal*" "' . $prefix . 'swpext*" --format=csv';

FORCE_SSL_ADMIN

Can we add the above constant to the wp-config templates for Development and Production to meet the acceptance criteria of our user stories?

HTTP user not compatible with all environments

These values were not working on my server because the user didn't exists. I changed the values to `whoami` which fixed my issue and seems to be multi applicable.

Perhaps getting these values from the config would be a nice addition.

PackerWP/deploy.php

Lines 128 to 129 in d9980aa

	set('http_user', 'www-data');
	set('http_group', 'www-data');

Local backup has wrong name

Prior to pulling a remote database down from a target host, a local db backup is created but it has the wrong stage name included in the filename.

ACF Block Search Replace - Missing URLs

I noticed today that when pulling and pushing DBs with the latest version of PackerWP (v1.2.5), it's missing URLs that are included in ACF gutenberg blocks.

Ash spotted the issue was because of the escaping in the search-replace command we use to target these specific URLs. This can be fixed by updated line 289 in tasks/db.php with the following:
"wp search-replace --quiet --all-tables '$from' '$to'",

Search and replace fails on ACF blocks

When moving the database between environments the wp search-replace command does not work for encoded urls within Gutenberg blocks.

ACF escape all of the url values that are stored in the database so https://mysite.com/foo becomes https:\/\/mysite.com\/foo

There is an open WP_CLI issue regarding the problem here: wp-cli/wp-cli#5293

As an iterim fix we can add additional find and replace commands to db.php for example:

// Task = db:import:remote
run('wp search-replace --report-changed-only --all-tables "' . addslashes(get('local_url')) . '" "' . addslashes(get('stage_url')) . '"', [
    'tty' => true,
    'timeout' => null
]);

// Task = db:import:local
runLocally('wp search-replace --report-changed-only --all-tables "' . addslashes(get('stage_url')) . '" "' . addslashes(get('local_url')) . '"', [
    'tty' => true,
    'timeout' => null
]);

Notice that each of the find and replace values are wrapped in double quotes as per the above WP-CLI issue

This does not seem to be a WordPress installation

Running "dep setup-local-wp" in the root of the project folder results with:

Error: This does not seem to be a WordPress installation.
The used path is: /Users/user/Sites/project-folder/
Pass --path=path/to/wordpress or run wp core download.

In Process.php line 250:
The command "tail -n+2 '/Users/user/Sites/project-folder/vendor/mixd/packerwp/config/templates/extras.php' | /usr/local/bin/wp config create --dbname="localdb_dev" \
--dbuser="root" \
--dbpass="root" \
--dbhost="localhost:/Applications/MAMP/tmp/mysql/mysql.sock" \
--locale="en_GB" \
--skip-check \
--force \
--extra-php" failed.

Exit Code: 1(General error)

Working directory: /Users/user/Sites/project-folder

Output:

Error Output:

This error goes away if I put deploy.json in the wordpress directory and run dep setup-local-wp though then there is another error:
The command "mv '/Users/user/Sites/project-folder/wordpress/wordpress/wp-config.php' '/Users/user/Sites/project-folder/wordpress/wp-config.php'" failed.

My project folder structure:
Root folder: /Users/user/Sites/project-folder/
deploy.json location: /Users/user/Sites/project-folder/
Wordpress files: /Users/user/Sites/project-folder/wordpress

PHP Version selection

I am currently trying to set up the script to run on our production server, however, the command line version of PHP is not compatible with the composer packages used, so I needed to select the PHP version manually.

It's probably a good idea to make an optional config option to select the PHP binary per environment.

Our production server is running DirectAdmin which supports multiple PHP versions to be installed and used on a per user basis. However, for the command line, the first version in the list will always be used, not the one connected to the user. The user I'm trying this on is on PHP 7.2 and the first version of our server is 7.0. We can't change this first version because of the possibility of breaking a lot of websites which we would then have to fix manually.

My problem was solved by adding this, but I'm sure there is a nicer implementation to be thought of:

if($stage == 'production') {
  set('bin/php', function () {
    return '/usr/local/bin/php74';
  });
}

running setup-local-wp gives an incorrect success message if the database already exists

If you have already migrated a database locally and you run dep setup-local-wp you get a message similar to this:

=========================================================================
WordPress has successfully been installed. Here are your login details:
Username:       xxx
Password:       xxxxxx
Email address:  [email protected]
Log in at:      https://xxx.local/wordpress/wp-admin
=========================================================================

Hover the above message is innacurate because the username and password above dont actually get setup instead Packer appears to use the previously defined user account credentials and doent create the above account. In my isntance the account created already existed so perhaps that narrows it down a little.

Improved Database Naming

I propose that we evolve the naming of the database backups so they're not just named after date but also include the environment too - when looking at DB backups it's not immediately obvious which are from staging and production environments.

I'd suggest:

{env_name}-Ymdhis

Move optional tasks into contrib folder

Having optional tasks like for running npm scripts in a contrib folder helps to keep the base deployment tasks maintainable and consistent and ensures contrib tasks can be hotswapped between projects

Project structure and location

Currently Packer is bundled as part of the NHS Framework and this includes the .env file but also the config folder that contains the tasks.

This creates a hard constraint between the version of the NHS Framework and also the Version of PackerWP. Can we explore packer becoming a composer package itself and only the config file being present in the theme. That way we can change the packer version independently.

Remote search-replace runs locally instead

PackerWP/config/tasks/db.php

Lines 118 to 152 in b53f634

	task('db:rewrite:remote', function () {
	$stage = get('stage');
	$wp_config = getconfig();
	$env_config = getenvbag($stage);
	$from = $wp_config['wp_home_url'];
	$to = $env_config['wp_home_url'];
	writeln('<info>Processing URL replacements</info>');
	writeln("<comment>$from → $to</comment>");
	runLocally(
	"wp search-replace --report-changed-only --all-tables $from $to",
	[
	'tty' => true,
	'timeout' => null
	]
	);
	if (!empty($wp_config['rewrite'])) {
	foreach ($wp_config['rewrite'] as $ruleset) {
	$from = $ruleset['local'];
	$to = $ruleset[$stage];
	writeln("<comment>$from → $to</comment>");
	runLocally(
	"wp search-replace --report-changed-only --all-tables $from $to",
	[
	'tty' => true,
	'timeout' => null
	]
	);
	}
	}
	})->setPrivate();

In our remote DB rewrite task, we perform a search-replace to correct the URLs from the local environment on the remote environment. However on line 128 and 143 you'll see we're using runLocally instead of run to perform the search-replace task which means it's run on the wrong environment.

Needs more verbose DB error messages

I came a cross a slight issue earlier and got the following message:

Database does not exist. Unable to continue

It isnt clear which DB doesn't exist, is it possible to specify which DB doesnt exist?

Feature request - Support for none 200 OK status codes in successful deployment

Packer currently correctly gives each user a warning when force login is enabled because the statuscode of the page is indeed returned as a 302.

The plugin currently spits out a warning:

    ========================================================================
        WARNING: 'wp_home_url' for 'staging' may not be exactly right!
        https://lthtintranet.mixd.co.uk responded with HTTP/2 302
    ========================================================================

This is quite jarring the first time you see it and is indicative of an underlying problem.

I propsoe that instead of this warning and then an option to rollback we define the expected HTTP status code (currently I believe we only count successes of 200 OK responses) but this issue indicates that there is scope to declare additional paramaters into Packer WP to define the expected behaviour of the heartbeat check - perhaps we can define that we are looking for a 302 response and that the location is https://mysite.com/wordpress/wp-login.php?redirect_to=https%3A%2F%2Fmysite.com%2F

Documentation

We need some detailed versbose documentation writing for Packer WP that covers the server side setup and has a Q and A section, plus a common problems section.

Check CSS through the CSS validator

There are a few instances of invalid css so it would be worth putting the css through:
https://jigsaw.w3.org/css-validator/validator