canthis / oc-chunk-import

Experimental implementation of Chunk Importer for October CMS. Based on October's original ImportExport behavior.

October CMS Chunk Import with progress bar

Experimental implementation of Chunk Importer for October CMS. Based on October's original ImportExport behavior.

Usage

This plugin comes with test models and controllers, so it's up and ready for testing. Take Test CSV file from /assets folder and try to import some data.

Now import process is splitted into chunks. Each chunk processes 50 records at a time, each chunk is a new AJAX request, and that's how progress bar is being updated.

There's also a Import Log side menu, where You can find user, who imported particular file, imported file name and date it was imported at.

oc-chunk-import's Issues

No result messages returned, statics going haywire

Hi,

Thanks! This is definitely a hugely useful enhancement to OctoberCMS.

I had a requirement for processing large imports (each row of which created multiple objects, so relatively slow running as well), which was not great with the standard import. I cut your plugin down to just the common code (removing your test data objects) and set my import up to use it.

The core functionality works well, but the data isn't being returned to the progress dialogue correctly - created, updated and error messages are not returned at all (not populated in the returned JSON), and while the stats at the top show errors in the chunk that they first occur, when the subsequent chunk returns the errors number disappears (though they are still reflected in the difference between processed and created - I don't have updates in this import, so I don't know if that is also affected).

The total rows processed is also slightly less than the total rows in my CSV (2349 reported versus 2353 rows of data in the CSV) - I've not worked out yet whether any rows were actually not processed, or whether it's simply the statistic that's wrong.

Hopefully this is helpful feedback!