GithubHelp home page GithubHelp logo

adrianbj / processmigrator Goto Github PK

View Code? Open in Web Editor NEW
29.0 7.0 2.0 285 KB

ProcessWire module that facilitates automated migration and sharing of page trees along with their templates and fields.

License: GNU General Public License v2.0

PHP 100.00%
processwire migration import export

processmigrator's Introduction

ProcessMigrator

Allows automated migration and sharing of page tree content along with their templates and fields via JSON files. These JSON files can be imported into another PW installation to recreate the entire structure and content, including files and images.

It takes care of replicating all the pages, as well as creating any templates and fields that are needed.

You can also use this module to share page lists. "Page Lists" are a page tree (parent and children) that store selctor values for a Page fieldtype. An example would be a list of countries that would be used to populate a countried drop-down select field. The fields might include: Country Name, 2-digit code, 3-digit code, number code. There is a repositories of page lists available from: https://github.com/adrianbj/ProcessWirePageLists and these can be added to your site from a dropdown select in this migrator module.

You can also use 3rd party converters to migrate content from other sources. So far we have one for Wordpress, thanks to Nico: https://github.com/NicoKnoll/MigratorWordpress

How to install

The easiest way to install, is to go to Modules > Site > Add New and in the "Add Module from URL" option, enter: https://github.com/adrianbj/ProcessMigrator/archive/master.zip

Usage

Go to the Setup Page > Migrator and follow the prompts.

What it can migrate

Fields, templates, and page content for all field types including:

  • All standard field types, including RTE, and decoding of links modified by the PageLinkAbstractor module and abstracting again on the destination PW install.
  • File/Image/CropImage fields including the actual files/images/thumbnails, and all other variations
  • All Profields field types: Table, PageTable, Multiplier, and Texareas
  • All? custom field types
  • Repeaters fields and all their required fields, templates, and content including files/images
  • Page fields (and the pages, templates, and fields that make up their selectable pages)
  • Multilanguage versions of field content, page names and page titles
  • Templates (including Access, Family, URL and other settings) and the template .php files. It even grabs the appropriate file if you are using the "Alternate Template Filename" setting. NB the templates directory on the destination PW installation must be writable for these to be imported.

Files/images/template files and the json structure/data file are exported in a zip file which is then imported into the destination PW install.

So, you could build sections of content on a local dev PW installation, export it, and then with a couple of clicks import everything into the live PW installation.

Notes

  • It supports multi-language fields, but you should make sure to have language support already installed on the destination installation before running the import
  • It supports links in RTE fields that have been converted with the PageLinkAbstractor module - make sure the module is installed on the destination server before running the import

Outstanding Issues

Some outstanding issues that I hope to get to shortly:

  • Need to support images inserted from a different page into an RTE field
  • Rewrite any references to page ids, eg $pages->get(xxxx) in template .php files so they will be converted to the correct id on the destination installation.
  • Need to look into the new core link abstractor that was added to PW 2.4 and see how to handle those links compared to the PageLinkAbstractor module.
  • Need to add checks so that existing template php files are not overwritten (or give the option to choose)
  • Might need to override PHP max_execution_time and other settings for larger exports and maybe chunk out zipping of all images to prevent memory issues on larger exports.

Support

https://processwire.com/talk/topic/8660-migrator

License

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

(See included LICENSE file for full license text.)

processmigrator's People

Contributors

adrianbj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

lisandi dojovader

processmigrator's Issues

Issue importing a large site, can not define the issue myself.

I think the issue might have to do with images being loaded before a page is save, however this is just a guess based on function names. Here is some more information

Reason for suspission

In the linked topic soma talks about saving a page before you can load images as the files where the images will be saved to wont exist.

Running the importer a second time by reloading the error page successfully loads the page with no errors.

https://processwire.com/talk/topic/352-creating-pages-via-api/

Stack Trace

(ProcessInstall replacing directory of the project)

( ! ) Notice: Trying to get property of non-object in ProcessInstall/site/modules/ProcessMigrator/ProcessMigrator.module on line 1568
Call Stack

Time Memory Function Location

1 0.0006 292368 {main}( ) .../index.php:0
2 0.0823 10787240 execute ( ) .../index.php:214
3 0.0823 10787592 Wire->__call( ) .../index.php:214
4 0.0823 10787592 Wire->runHooks( ) .../Wire.php:344
5 0.0823 10790248 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
6 0.0823 10790584 ProcessPageView->___execute( ) .../Wire.php:389
7 0.0979 11858528 render ( ) .../ProcessPageView.module:171
8 0.0979 11858712 Wire->__call( ) .../ProcessPageView.module:171
9 0.0979 11858712 Wire->runHooks( ) .../Wire.php:344
10 0.0980 11863696 renderPage ( ) .../Wire.php:411
11 0.0980 11864048 Wire->__call( ) .../Wire.php:411
12 0.0980 11864048 Wire->runHooks( ) .../Wire.php:344
13 0.0980 11866176 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
14 0.0980 11866512 PageRender->___renderPage( ) .../Wire.php:389
15 0.0986 11933672 render ( ) .../PageRender.module:356
16 0.0986 11933856 Wire->__call( ) .../PageRender.module:356
17 0.0986 11933856 Wire->runHooks( ) .../Wire.php:344
18 0.0986 11935984 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
19 0.0986 11936152 TemplateFile->___render( ) .../Wire.php:389
20 0.0987 11962144 require( 'ProcessInstall/site/templates/admin.php' ) .../TemplateFile.php:169
21 0.0987 11963816 require( 'ProcessInstall/wire/modules/AdminTheme/AdminThemeDefault/controller.php' ) .../admin.php:13
22 0.0990 11998728 require( 'ProcessInstall/wire/core/admin.php' ) .../controller.php:13
23 0.1018 12337720 execute ( ) .../admin.php:85
24 0.1018 12337904 Wire->__call( ) .../admin.php:85
25 0.1018 12337904 Wire->runHooks( ) .../Wire.php:344
26 0.1019 12340032 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
27 0.1019 12340200 ProcessController->___execute( ) .../Wire.php:389
28 0.1061 13118384 executeImport ( ) .../ProcessController.php:204
29 0.1061 13118568 Wire->__call( ) .../ProcessController.php:204
30 0.1061 13118568 Wire->runHooks( ) .../Wire.php:344
31 0.1061 13120704 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
32 0.1061 13120872 ProcessMigrator->___executeImport( ) .../Wire.php:389
33 1.5922 15090216 ProcessMigrator->processImportForm2( ) .../ProcessMigrator.module:139

( ! ) Fatal error: Call to a member function get() on a non-object in ProcessInstall/site/modules/ProcessMigrator/ProcessMigrator.module on line 2583
Call Stack

Time Memory Function Location

1 0.0006 292368 {main}( ) .../index.php:0
2 0.0823 10787240 execute ( ) .../index.php:214
3 0.0823 10787592 Wire->__call( ) .../index.php:214
4 0.0823 10787592 Wire->runHooks( ) .../Wire.php:344
5 0.0823 10790248 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
6 0.0823 10790584 ProcessPageView->___execute( ) .../Wire.php:389
7 0.0979 11858528 render ( ) .../ProcessPageView.module:171
8 0.0979 11858712 Wire->__call( ) .../ProcessPageView.module:171
9 0.0979 11858712 Wire->runHooks( ) .../Wire.php:344
10 0.0980 11863696 renderPage ( ) .../Wire.php:411
11 0.0980 11864048 Wire->__call( ) .../Wire.php:411
12 0.0980 11864048 Wire->runHooks( ) .../Wire.php:344
13 0.0980 11866176 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
14 0.0980 11866512 PageRender->___renderPage( ) .../Wire.php:389
15 0.0986 11933672 render ( ) .../PageRender.module:356
16 0.0986 11933856 Wire->__call( ) .../PageRender.module:356
17 0.0986 11933856 Wire->runHooks( ) .../Wire.php:344
18 0.0986 11935984 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
19 0.0986 11936152 TemplateFile->___render( ) .../Wire.php:389
20 0.0987 11962144 require( 'ProcessInstall/site/templates/admin.php' ) .../TemplateFile.php:169
21 0.0987 11963816 require( 'ProcessInstall/wire/modules/AdminTheme/AdminThemeDefault/controller.php' ) .../admin.php:13
22 0.0990 11998728 require( 'ProcessInstall/wire/core/admin.php' ) .../controller.php:13
23 0.1018 12337720 execute ( ) .../admin.php:85
24 0.1018 12337904 Wire->__call( ) .../admin.php:85
25 0.1018 12337904 Wire->runHooks( ) .../Wire.php:344
26 0.1019 12340032 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
27 0.1019 12340200 ProcessController->___execute( ) .../Wire.php:389
28 0.1061 13118384 executeImport ( ) .../ProcessController.php:204
29 0.1061 13118568 Wire->__call( ) .../ProcessController.php:204
30 0.1061 13118568 Wire->runHooks( ) .../Wire.php:344
31 0.1061 13120704 call_user_func_array:{ProcessInstall/wire/core/Wire.php:389} ( ) .../Wire.php:389
32 0.1061 13120872 ProcessMigrator->___executeImport( ) .../Wire.php:389
33 1.5922 15090216 ProcessMigrator->processImportForm2( ) .../ProcessMigrator.module:139
34 11.6868 54486296 ProcessMigrator->idImagePath( ) .../ProcessMigrator.module:1587

Error Message at the bottom of the stack trace:

Error: Call to a member function get() on a non-object (line 2583 of ProcessInstall/site/modules/ProcessMigrator/ProcessMigrator.module)
This error message was shown because site is in debug mode ($config->debug = true; in /site/config.php). Error has been logged.

MigratorWordpress import fails

I am trying to import a wordpress-xml file with MigratorWordpress. The fields and templates are created successfully, but no pages are imported/created.
If using a 'Import Type' of 'Append' the import silently fails, when using 'Overwrite' or 'Replace' I get this error:
Error: Uncaught Error: Call to a member function get() on null in /xxx/site/modules/ProcessMigrator/ProcessMigrator.module:1450

Tested on fresh PW 2.7.3 and 3.0.16 with PHP7.

date () issue

Warning: date() expects parameter 2 to be long, string given in C:\inetpub\wwwroot\PW1\wire\modules\Fieldtype\FieldtypeDatetime.module on line 401

Error due to node_modules and too many sub-directories

I have a Gulp-based frontend workflow, and as a result, a "node_modules" folder appears in my /site/templates/ folder, which is normal.

Some Node modules have many many nested subdirectories, which sometimes hits a limitation on Windows. When trying to Export using ProcessMigrator, I get an error which suggests that may be the issue:

 ProcessWire: ProcessMigrator: RecursiveDirectoryIterator::__construct(C:/Users/jlahijani/Projects/mysite.com/site/templates\node_modules\gulp-imagemin\node_modules\imagemin\node_modules\imagemin-gifsicle\node_modules\gifsicle\node_modules\bin-build\node_modules\decompress\node_modules\buffer-to-vinyl\node_modules\uuid\benchmark,C:/Users/jlahijani/Projects/mysite.com/site/templates\node_modules\gulp-imagemin\node_modules\imagemin\node_modules\imagemin-gifsicle\node_modules\gifsicle\node_modules\bin-build\node_modules\decompress\node_modules\buffer-to-vinyl\node_modules\uuid\benchmark): The system cannot find the path specified. (code: 3)
DEBUG MODE BACKTRACE ($config->debug == true):
#0 [internal function]: RecursiveDirectoryIterator->__construct('C:/Users/jlahij...', 4096)
#1 C:\Users\jlahijani\Projects\mysite.com\site\modules\ProcessMigrator\ProcessMigrator.module(280): RecursiveDirectoryIterator->getChildren()
#2 C:\Users\jlahijani\Projects\mysite.com\site\modules\ProcessMigrator\ProcessMigrator.module(120): ProcessMigrator->buildExportForm2()
#3 [internal function]: ProcessMigrator->___executeExport()
#4 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(397): call_user_func_array(Array, Array)
#5 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('executeExport', Array)
#6 C:\Users\jlahijani\Projects\mysite.com\wire\core\ProcessController.php(236): Wire->__call('executeExport', Array)
#7 C:\Users\jlahijani\Projects\mysite.com\wire\core\ProcessController.php(236): ProcessMigrator->executeExport()
#8 [internal function]: ProcessController->___execute()
#9 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(397): call_user_func_array(Array, Array)
#10 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('execute', Array)
#11 C:\Users\jlahijani\Projects\mysite.com\wire\core\admin.php(93): Wire->__call('execute', Array)
#12 C:\Users\jlahijani\Projects\mysite.com\wire\core\admin.php(93): ProcessController->execute()
#13 C:\Users\jlahijani\Projects\mysite.com\wire\modules\AdminTheme\AdminThemeDefault\controller.php(13): require('C:\\Users\\jlahij...')
#14 C:\Users\jlahijani\Projects\mysite.com\site\templates\admin.php(15): require('C:\\Users\\jlahij...')
#15 C:\Users\jlahijani\Projects\mysite.com\wire\core\TemplateFile.php(182): require('C:\\Users\\jlahij...')
#16 [internal function]: TemplateFile->___render()
#17 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(397): call_user_func_array(Array, Array)
#18 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('render', Array)
#19 C:\Users\jlahijani\Projects\mysite.com\wire\modules\PageRender.module(422): Wire->__call('render', Array)
#20 C:\Users\jlahijani\Projects\mysite.com\wire\modules\PageRender.module(422): TemplateFile->render()
#21 [internal function]: PageRender->___renderPage(Object(HookEvent))
#22 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(397): call_user_func_array(Array, Array)
#23 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('renderPage', Array)
#24 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(453): Wire->__call('renderPage', Array)
#25 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(453): PageRender->renderPage(Object(HookEvent))
#26 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('render', Array)
#27 C:\Users\jlahijani\Projects\mysite.com\wire\modules\Process\ProcessPageView.module(187): Wire->__call('render', Array)
#28 C:\Users\jlahijani\Projects\mysite.com\wire\modules\Process\ProcessPageView.module(187): Page->render()
#29 [internal function]: ProcessPageView->___execute(true)
#30 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(397): call_user_func_array(Array, Array)
#31 C:\Users\jlahijani\Projects\mysite.com\wire\core\Wire.php(332): Wire->runHooks('execute', Array)
#32 C:\Users\jlahijani\Projects\mysite.com\index.php(241): Wire->__call('execute', Array)
#33 C:\Users\jlahijani\Projects\mysite.com\index.php(241): ProcessPageView->execute(true)
#34 {main}

Perhaps completely ignoring "node_modules" would be a good idea, or at least a configuration option to. This could also apply to "bower_components" when using bower.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.