Vanilla 1 to Vanilla 2 Import about vanilla

[saladyears]

Yes, please! :)

[linc]

The recent HTML/CSS changes messed up the alignment of the import form and labels are now the same size as H2.

[picklit]

Negative votes? Why wouldn't there be a Vanilla 1 -> Vanilla 2 import?

[linc]

Votes are meaningless, really. This is a priority.

[adamramadhan]

this is an must-have. good choice.

[barnaclebarnes]

So is there no way to upgrade from Vanilla 1 to 2 right now? I've tried to install Vanilla 2 and point it to an existing v1 database but can't get things working.

[linc]

Right now, your best bet is to wait a little longer because the import/export mechanism is getting finished up and you'll end up with a much better result using the final version rather than the current work-in-progress.

[tburry]

RC3 has the importer exporter now. Would love it if people could test it a bit. There is documentation here: http://vanillaforums.com/blog/help-topics/importing-data/

[kerphi]

I tried the RC3 importer with the exported data generated by http://vanillaforums.com/blog/help-topics/importing-data/

Export works but import blocks on the "Authenticate Admin User" step : I tried to both enter my local vanilla2 admin account and my old vanilla1 admin account. Someone know how to correct or skip this step ?

[scottsmith]

I created a new job here relating to this: http://github.com/vanillaforums/Garden/issues#issue/432

[barnaclebarnes]

Mine won't allow the file to be uploaded. It said the max file size exceeds servers max. Interestingly the error changes based on what I set max_file_size at:

When post_max_size = 8M

The file is larger than the server's maximum file size. (8M)

When post_max_size = 14M

The file is larger than the server's maximum file size. (2M)

The file is 10MB but I am wondering where it is getting the 2MB size from. Is there another file size setting to look at?

Also is it possible to add a command line importer? It seems easier than having to ship large files over HTTP if we have access to our servers.

[tburry]

@barnaclebarnes: There is an application config setting called Garden.Upload.MaxFileSize which you also must change.

You can also just put the exported file in your uploads folder and browse to the page. It will show up with an option.

[tburry]

Okay. I found the issue and some others and uploaded fixes. The latest branch has the fixes here:
http://github.com/vanillaforums/Garden/tree/22787e96f9b98fbfef345f3aba563e734dadd722.
You can try replacing just dashboard/models/class.importmodel.php if you want to test the fixes.

Also, I did find some problems with line endings and different operating systems. I don't thing this affects .gz uploads, but if you are having problems you might want to re-export with the latest vanilla2export.php here: http://vanillaforums.com/porter/vanilla2export.php.gz.

[barnaclebarnes]

@tburry I hadn't changed that setting. It is at the default. I found another php setting I needed to change as well:

upload_max_filesize = 15M

Now my next issue. I am trying the Overwrite this forum option. What email/password should I use here? I'm not sure if this creates a new user or has to be an admin user from the old database or an admin user from the new database.

No matter what I try I get this error:

"Sorry no account could be found related to the email and password you entered"

I'm using the latest version from GitHub as posted just above.

[kerphi]

Thank you tburry, the new dashboard/models/class.importmodel.php is working for my data without having to regenerate the export. I have to check my categories/users's roles now because I saw that they were reset.

[mwalliser]

Tried the export tool to export from Vanilla 1, which created the file however when doing so, the following showed up on screen:

Notice: Table 'vanilla.user' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.role' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.user' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.category' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.discussion' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.comment' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.userdiscussionwatch' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.comment' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.comment' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528

Notice: Table 'vanilla.comment' doesn't exist in C:\Inetpub\eNews\Vanilla\vanilla2export.php on line 528
Vanilla

Vanilla Porter

Export Started: 2010-07-14 10:32:36

Exported Table: User (0 rows, 00:00.00)

Exported Table: Role (0 rows, 00:00.00)

Exported Table: UserRole (0 rows, 00:00.01)

Exported Table: Category (0 rows, 00:00.00)

Exported Table: Discussion (0 rows, 00:00.00)

Exported Table: Comment (0 rows, 00:00.01)

Exported Table: UserDiscussion (0 rows, 00:00.00)

Exported Table: Conversation (0 rows, 00:00.00)

Exported Table: ConversationMessage (0 rows, 00:00.00)

Exported Table: UserConversation (0 rows, 00:00.01)

Export Completed: 2010-07-14 10:32:36

Elapsed Time: 00:00.05
Download export_Vanilla_2010-07-14_103236.txt.gz

I figured that since it created the file, that I'd try it anyway. So when I went to import the data into my Vanilla 2 installation I got the following:

FATAL ERROR IN: Gdn_Database.Query();
"Table 'vanilla.gdn_zuser' doesn't exist" truncate table GDN_zUser; LOCATION: C:\Inetpub\eNews\Vanilla2\library\database\class.database.php > 232: $PDOStatement = $this->Connection()->query($Sql); > 233: } > 234: > 235: if ($PDOStatement === FALSE) { >>> 236: trigger_error(ErrorMessage($this->GetPDOErrorMessage($this->Connection()->errorInfo()), $this->ClassName, 'Query', $Sql), E_USER_ERROR); > 237: } > 238: > 239: $ReturnType = GetValue('ReturnType', $Options); > 240: BACKTRACE: [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php] PHP::Gdn_ErrorHandler(); [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php 236] PHP::trigger_error(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 734] Gdn_Database->Query(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 698] ImportModel->LoadTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 698] ImportModel->LoadUserTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 967] PHP::call_user_func(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\controllers\class.importcontroller.php 59] ImportModel->RunStep(); [C:\Inetpub\eNews\Vanilla2\library\core\class.dispatcher.php 264] ImportController->Go(); [C:\Inetpub\eNews\Vanilla2\index.php 39] Gdn_Dispatcher->Dispatch();

[tburry]

@barnaclebarnes, You need to log in with the email and password in the old database. It is possible that you experienced the bug that was addressed yesterday so you might want to try the latest ImportModel. My previous comment above has the instructions.

@mwalliser, You didn't enter an appropriate database prefix. However, the export/import should be a bit more robust. I've programmed a check in the exporter and I'll address the fatal error in the import.

[mwalliser]

Thanks @tburry. I didn't enter a database prefix with the export of the DB because it said it wasn't required. When I re-exported the data with the prefix, I was able to export without errors. However, when I tried to import that file into the Vanilla 2 installation I still get a fatal error (different from the previous one)...

FATAL ERROR IN: Gdn_Database.Query();
"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'character set utf8 columns terminated by ',' optionally encl' at line 2" load data infile 'C:\Inetpub\eNews\Vanilla2/uploads/import/User.txt' into table GDN_zUser character set utf8 columns terminated by ',' optionally enclosed by '"' escaped by '' lines terminated by '\n' ignore 1 lines LOCATION: C:\Inetpub\eNews\Vanilla2\library\database\class.database.php > 232: $PDOStatement = $this->Connection()->query($Sql); > 233: } > 234: > 235: if ($PDOStatement === FALSE) { >>> 236: trigger_error(ErrorMessage($this->GetPDOErrorMessage($this->Connection()->errorInfo()), $this->ClassName, 'Query', $Sql), E_USER_ERROR); > 237: } > 238: > 239: $ReturnType = GetValue('ReturnType', $Options); > 240: BACKTRACE: [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php] PHP::Gdn_ErrorHandler(); [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php 236] PHP::trigger_error(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 996] Gdn_Database->Query(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 763] ImportModel->Query(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 738] ImportModel->_LoadTableOnSameServer(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 698] ImportModel->LoadTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 698] ImportModel->LoadUserTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 967] PHP::call_user_func(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\controllers\class.importcontroller.php 59] ImportModel->RunStep(); [C:\Inetpub\eNews\Vanilla2\library\core\class.dispatcher.php 264] ImportController->Go(); [C:\Inetpub\eNews\Vanilla2\index.php 39] Gdn_Dispatcher->Dispatch();

[tburry]

@mwalliser, I've done the following to address this:

1. I've made the description of the database a bit more clear.
2. If you leave the database prefix blank then there will be an error message with a best guess.
3. There should now be a bit more error checking on the import if you import a bogus file.

All these are in the latest code (not rc3).

[mwalliser]

@tburry Thanks! I just saw the post on the forums about the latest update and am downloading now. Hopefully all goes well and I don't have anything new to report. :)

[mwalliser]

@tburry sadly, I get the same error still with the latest build. Maybe there's a problem with the exporter file and not necessarily the import?? Is there a more updated version than the one that is linked above?

[tburry]

We've got a link to always the latest source code in http://vanillaforums.org/download/nightly. I just did an install with this and the latest vanilla2export.php.gz and both gave the proper errors I gave above.

Can you give it another go? Sorry about the roundabout. I really appreciate your help.

[fmimoso]

Having the same problem.

Ran Vanilla 1 exporter
Installed Vanilla 2
Uploaded export file to /uploads
Site settings -> Import

Here you should be clear wether you want the USERNAME or the EMAIL - in the caption it asks for email, in the introduction it says username: which one?

Nevertheless, I've tried every single combination that I've remembered and:

• first I would receive the following notice http://i31.tinypic.com/eued5x.jpg
• after some tries, nothing more than http://i29.tinypic.com/20jmwk5.png

More info:
http://i32.tinypic.com/2dqttm1.png
http://i30.tinypic.com/5jxobb.png

[mwalliser]

Tried again... making sure I had downloaded the latest build and the latest export file and came up with the same error again...

FATAL ERROR IN: Gdn_Database.Query();
"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'character set utf8 columns terminated by ',' optionally encl' at line 2" load data infile 'C:\Inetpub\eNews\Vanilla2/uploads/import/User.txt' into table GDN_zUser character set utf8 columns terminated by ',' optionally enclosed by '"' escaped by '' lines terminated by '\n' ignore 1 lines LOCATION: C:\Inetpub\eNews\Vanilla2\library\database\class.database.php > 232: $PDOStatement = $this->Connection()->query($Sql); > 233: } > 234: > 235: if ($PDOStatement === FALSE) { >>> 236: trigger_error(ErrorMessage($this->GetPDOErrorMessage($this->Connection()->errorInfo()), $this->ClassName, 'Query', $Sql), E_USER_ERROR); > 237: } > 238: > 239: $ReturnType = GetValue('ReturnType', $Options); > 240: BACKTRACE: [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php] PHP::Gdn_ErrorHandler(); [C:\Inetpub\eNews\Vanilla2\library\database\class.database.php 236] PHP::trigger_error(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 1074] Gdn_Database->Query(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 792] ImportModel->Query(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 763] ImportModel->_LoadTableOnSameServer(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 718] ImportModel->LoadTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 718] ImportModel->LoadUserTable(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\models\class.importmodel.php 1045] PHP::call_user_func(); [C:\Inetpub\eNews\Vanilla2\applications\dashboard\controllers\class.importcontroller.php 59] ImportModel->RunStep(); [C:\Inetpub\eNews\Vanilla2\library\core\class.dispatcher.php 270] ImportController->Go(); [C:\Inetpub\eNews\Vanilla2\index.php 38] Gdn_Dispatcher->Dispatch();

[tburry]

@mwalliser can I get your exact mysql version? You can use this command:

select version() as version;

[mwalliser]

@tburry it's 5.0.27-community-nt

[tburry]

@mvalliser, I put a version check in the export, but if you are using MySQL 4+ and can upgrade to MySQL 5+ then I recommend that you do.

[tburry]

Sorry, didn't read your comment. I will have to check the support for that version and my load data infile syntax.

[tburry]

@mvalliser, @fmimoso it looks like you both have different problems. I've changed the way the table loading is determined which should hopefully solve both your problems.

If you can grab the latest code and see if the import works on your servers then that would be great.

[fmimoso]

Tested it right now and... it works! :)

[fmimoso]

Another problem... My database's colation is Latin1 as you can see by the above screenshots I posted. In conf.php I have: $Configuration['CHARSET'] = 'utf-8';
Result in Vanilla2 after import: scrambled international characters, for example - canalização where it should be canalização.
In the database (through phpMyAdmin) it displays canalização both in Vanilla1 database and Vanilla2.
Any ideas on how to prevent this?

[mwalliser]

I was so close... made it farther through the import process than I had before... now it gets to the "Insert User Table" step and I get this error:

Fatal error: Nesting level too deep - recursive dependency? in C:\Inetpub\eNews\Vanilla2\library\core\class.authenticator.php on line 115

[tburry]

@fmimoso, A couple of questions:

1. Was your vanilla database backed up and restored using mysql_dump?
2. Do the characters display correctly or incorrectly in Vanilla 1?

[tburry]

@mwalliser, there is a fix up. If you are only pulling single files the fix is in: library/core/class.authenticator.php.

[mwalliser]

Great! So where do I get this fix? I looked at the source for that file here on GitHub but the last update was a few days ago... I've downloaded the whole thing since then...

Oh, (please excuse my ignorance) what exactly do you mean my "only pulling single files"?

[tburry]

The file was changed, but maybe you were searching through the master branch which is the default view?

I gave you the one file with the fix in case you're uploading to an ftp server and only want to copy the one file rather than wait for everything to copy up.

[mwalliser]

To be safe I re-downloaded the entire thing at http://vanillaforums.org/download/nightly and tried again... same error as before.
Fatal error: Nesting level too deep - recursive dependency? in C:\Inetpub\eNews\Vanilla2\library\core\class.authenticator.php on line 115

[tburry]

Can you make sure that the file class.authenticator.php line 119 says "return;"? You are kind of stumping me.

[mwalliser]

It does...
:( sorry...

[tburry]

Are you on a particular webhost? Maybe we can try them ourselves.

[mwalliser]

No, it's our network here at work... hosted internally.

[tburry]

Okay, I've tried another push which I hope works, but you are stumping me.

[mwalliser]

IT FINALLY WORKED!!! :)
I don't know what you did... but THANK YOU!!

[tburry]

Dude, you're a trooper. Thanks so much for doing the testing.

[fmimoso]

1 - I think you're asking if I ever restored database from a mysql dump sometime before this import, right? I don't recall but I don't think so. I've gone through a server migration, nothing more. One important notice: this is a 2006 database, when Vanilla 1.0 was released.
2 - Correctly in Vanilla 1, incorrectly in Vanilla 2.

[tburry]

Okay @fmimoso. It sounds like you have a version of Vanilla 1 installed before it made the switch to utf8. This is something that I will look into supporting on the export side, but you can also update your Vanilla 1 installation.

One thing I'd say though is if you need those characters to be correct then it might not be possible to get them fixed after the data has been imported into Vanilla 2. I would make sure things look alright first.

There is a document that someone put together a while ago about switching to utf8 here: http://lussumo.com/docs/doku.php?id=vanilla:administrators:encodings

[tburry]

@fmimoso, Can you execute the following command and tell me what you see for the Collation column?

show table status like 'LUM_Comment';

[fmimoso]

http://i28.tinypic.com/op6x74.png

Regarding your previous message, I've read it but I'm not able to try it right now.

[tburry]

Okay @fmimoso, I've updated the vanilla2export.php.gz to detect encodings and have tried it on a Vanilla 1.0.1 install. If you can give it a try too when you get a chance that would be great.

(And don't worry about reading that long post.)

[fmimoso]

Once again, it works! :)

Thanks for your commitment. :)

[tburry]

Awesome. Were the characters fixed in the new import?

[fmimoso]

Yes. :)

[tburry]

Okay, I think I'm going to close this issue. I'm sure there will be more as we add more support for other forum exports, but I think its better to have those issues be a bit more targeted to specific bugs.

Thanks for all the help!