mozilla / mysql-patcher

INACTIVE - http://mzl.la/ghe-archive - A package/program to help patch MySql databases

mysql-patcher

A package/program to help patch MySql databases.

Synopsis

This is the simplest program that can work:

var path = require('path')

var mysql = require('mysql')
var patcher = require('mysql-patcher')

var options = {
  user       : 'user',
  database   : 'db',
  password   : 'password',
  dir        : path.join(__dirname, 'schema'),
  patchKey   : 'schema-patch-level',
  patchLevel : 4,
  filePrefix : 'patch',
  metaTable  : 'metadata',
  mysql      : mysql,
}

patcher.patch(options, function(err, res) {
  console.log('err:', err)
  console.log('res:', res)
})

Note: you should require mysql yourself and pass this to .patch() so that we're using the version you want, instead of us depending on mysql ourselves.

.patch(options)

The options are passed straight through to MySql, so you can provide any of the following:

• https://github.com/felixge/node-mysql#connection-options

Discussed below are some of more regular ones, but if not provided they will take the defaults specified on the mysql page (above):

• user : the user for the database (requires permission to create the database if needed)
• password : the password for the database
• host : the host for the database
• port : the port for the database
• socketPath : the socket (instead of host and port)
• database : the database name

Specific options for mysql-patcher:

• dir : string - the directory where the patch files live
• patchLevel : integer - the level to which the database should patched
• metaTable : string - the metaTable name
• patchKey : string - the name of the row in the metaTable which stores the current patch
• createDatabase : true/false - tries to create the database if it doesn't exist (default: false)
• reversePatchAllowed : true/false - allow reverse patching to take place (default: false)
• filePrefix : string - the patchfile prefix to look for e.g. patch-001-002.sql (default: 'patch')

Database Patch Files

All patch files should be named in the following format:

• <name>-<from>-<to>.sql
• e.g. patch-0001-0002.sql

This example is a patch file from level 1 to level 2.

Each database patch file should perform any queries they want first, then the last statement should set your patchKey value (in the metaTable) to the patch specified

Your Initial Patch

Your initial patch shouldn't do much except create the metaTable and set the patchKey row to be 1.

If you don't know what to do, copy and paste these two files for your initial forward and reverse patches:

e.g. Forward patch file : patch-00-01.sql

CREATE TABLE metadata (
  name VARCHAR(255) NOT NULL PRIMARY KEY,
  value VARCHAR(255) NOT NULL
) ENGINE=InnoDB;

INSERT INTO metadata SET name = 'schema-patch-level', value = '1';

e.g. Reverse patch file : patch-01-00.sql

DROP TABLE metadata;

Patches 2 and above

Once your initial patch has worked, each subsequent patch (both forward and reverse) should not try to insert the patch level, but instead update it:

e.g. Forward patch file : patch-01-02.sql

UPDATE metadata SET value = '2' WHERE name = 'schema-patch-level';

e.g. Reverse patch file : patch-02-01.sql

UPDATE metadata SET value = '1' WHERE name = 'schema-patch-level';

Changelog

Pending

• none

v0.7.0 - 2015-03-16

• added option 'filePrefix' to tighten which files are classed as patch files
• fixed a test related to access for an unknown user

v0.6.1 - 2015-02-02

• fixed up a test related to patch application

v0.6.0 - 2015-02-02

• added ability to create an instance of Patcher which can be controlled more succinctly

v0.5.1 - 2015-01-21

• fix patch direction error message

v0.5.0 - 2015-01-19

• fix error handling when the connection fails

v0.4.0 - 2014-11-14

• added a check between patches to make sure the patch level was incremented properly

License

Mozilla Public License v2

(Ends)

mysql-patcher's Issues

Check the patch level after each patch

In function applyPatches(), we should check the patch level after we have applied every patch. Currently this is not checked and it would make sense to do so.

Add example of `new Patcher()` to the readme

Ignore dot-files in the patch dir

If I try to run the patcher while I've got a patch file open in vim, I get this error:

Unknown file format: /Users/rfk/repos/mozilla/identity/fxa-auth-db-mysql/db/schema/.patch-008-009.sql.swp

It's trying to read the vim swapfile as a patch. I think we'd be pretty safe to apply unix tradition and ignore filenames that start with a dot. We could even explicitly filter only that with particular file extensions, in case people want to put e.g. a readme in the patch directory.

incorrect error message when direction is '-1'

This line https://github.com/chilts/mysql-patcher/blob/master/index.js#L220 assumes that direction is '1' (forward) when reporting a failure to find a match. e.g., if currentPatchLevel is 7, and nextPatchLevel is 6, the error will say "Patch from level 7 to 8 does not exist".

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

error running tests over case sensitivity of table names

When running the tests locally, I got:

  run an end to end test, with no error (to patch 0)
    ✓ There was no error when patching the database
    ✓ There was an error getting the database patch level
    ✓ No dbMetadata table
    ✓ Correct error number
    ✗ Correct message
     ---
       operator: equal
       expected: 'ER_NO_SUCH_TABLE: Table \'patcher.dbMetadata\' doesn\'t exist'
       actual:   'ER_NO_SUCH_TABLE: Table \'patcher.dbmetadata\' doesn\'t exist'
       at: Query._callback (/Users/jrgm/github/chilts/mysql-patcher/test/end-to-end.js:37:9)
     ...

The failure happened on osx (mysql version 5.6.19-log) where the default for lower_case_table_names is 2

mysql> show variables like '%case%';
show variables like '%case%';
+------------------------+-------+
| Variable_name          | Value |
+------------------------+-------+
| lower_case_file_system | ON    |
| lower_case_table_names | 2     |
+------------------------+-------+
2 rows in set (0.00 sec)

Add npm-shrinkwrap.json?

@chilts, let me know if you want me to submit a PR which adds dependencies+devDependencies in a shrinkwrap file.

Or close this and tell me to get lost, your call.

Fill in the CHANGELOG

It has v0.4.0 but nothing else yet.

Don't mutate the input options object

It's fairly likely that users of this lib will pass in their raw MySQL options object, so we should avoid mutating it. In particular, I notice that we set options.multipleStatements=true on the input options object and the calling code may not want that.

Since we have a dep on clone already, I suggest just cloning the options object up-front and doing our own mutations from there.

Improve error handling/reporting

As noted in mozilla/fxa-auth-db-server#105 (comment) there are currently paths where the callback is not called in case of error.