saurabhbatra96 / com.saurabhbatra96.civiimport

Import to any API for CiviCRM. Part of my GSoC project.

com.saurabhbatra96.civiimport

Import to any API for CiviCRM - Google Summer of Code

Disclaimer - This repo is not being actively maintained and remains in beta.

The aim

The aim of this project was to have an easy to use extension to import CiviCRM client data from a plethora of file formats (ex. CSV, Excel sheets) and also have support to import directly from Google Sheets. All this, using AngularJS on top to provide a better UX.

What has been accomplished

While the extension is not in a state that clients can use with no hassle, it currently serves as a prototype which, with some tweaks (to both UI and the back-end PHP code) can be shipped as a "Core" extension along with CiviCRM.

• Users can comfortably import from a CSV file.
• Users can comfortably import from Google Sheets.
• Strict validation and verification of data to be imported.
  • See: civicrm/civicrm-core#8624
• No need for a page refresh to change previously set parameters (thank you Angular!).

What needs to be done

• One of the main ideas behind this extension was to have a data-source interface which allowed imports from multiple file formats, currently this extension only supports CSV files.
• Google's Developer Console has this annoying (I know it's for security but that doesn't stop it from being a pain in the back) requirement where you have to list out allowed redirect URLs for your app. That means, basically anyone who's planning on using the extension has to manually insert their URL in Developer's Console, generate their own version of "client_secret.json" and paste it into their extension folder. We need to automate this!

com.saurabhbatra96.civiimport's Issues

Add some navigation links

Hey,

What were you thinking about adding WRT navigation links?

This is the function I used (although there may be a better way) https://github.com/eileenmcnaughton/nz.co.fuzion.csvimport/blob/master/csvimport.php#L78

Update version number and header check

This worked very well for me. I'm running 4.7.16. I had to update the Info.xml to version 4.7. Could definitely use a few tweaks if you were inclined:

• File Includes header/skip first row

• The Use Mapping button is kind of silly. The first time I ran through it, I didn't click on it because I had assumed that was to use an existing mapping or something. If I just entered the mapping, of course I want you to use it. I could see if that Use Mapping button was instead of the Next, but that isn't the case.

• I know I see in a lot of other importing tools the ability to specify date format. The lack of this wasn't an issue for me at all because I was simply using this as an option value importer and dates are rarely used (though I guess they could be).

This ran very fast and very smoothly once I understood the intricacies. For A GSOC project, I would rank this as excellent. Its darn good regardless; just a few tweaks and I would definitely upgrade it to a beta. ;)

Filter the entity list

It's probably work filtering the entity list by whether the entity has a create action - otherwise you get some invalid entities like 'SurveyRespondant' - which apart from anything else is a misspelling that has been annoying me for years.

Unfortunately the initial load is already a little slow & I can't think of a more efficient way to do this than to iterate through the actions & find out if they have a create action ('getactions'). Perhaps we should cache the results - e.g like

  $globalCache = CRM_Utils_Cache::singleton();
  $result = $globalCache->get($cacheKey);

NB I would kind of like to be able to import to the profile.submit method too - but that doesn't have to be in scope

error on clicking google import button

Error: [$interpolate:interr] http://errors.angularjs.org/1.3.20/$interpolate/interr?p0=%0A%20%20%7B%7Buploader%7D%7D%0A&p1=TypeError%3A%20cyclic%20object%20value
S/<@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:6:417
v@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:88:68
p/c<@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:109:415
h/<@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:108:90
Pe/this.$get</n.prototype.$digest@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:123:478
Pe/this.$get</n.prototype.$evalAsync/<@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:126:335
e@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:40:72
nf/n.defer/c<@http://d7-master.dev/sites/all/modules/civicrm/bower_components/angular/angular.min.js?ob7vlu:44:162
e/<()angular...?ob7vlu (line 102)
He/this.$get</<(a=Object { type="object"}, c=undefined)angular...?ob7vlu (line 76)
v()angular...?ob7vlu (line 88)
p/c<(c=Object { type="object"}, d=undefined)angular...?ob7vlu (line 109)
h/<(a=Object { type="object"})angular...?ob7vlu (line 108)
Pe/this.$get</n.prototype.$digest()angular...?ob7vlu (line 123)
Pe/this.$get</n.prototype.$evalAsync/<()angular...?ob7vlu (line 126)
e(a=Object { type="object"})angular...?ob7vlu (line 40)
nf/n.defer/c<()

Usability - grey out next button on first screen until a datasource is loaded

I noticed that I can browse and find a file and then click 'next' without uploading it - which leaves me with a confusing empty screen on the next screen. I wonder about greying out the 'next' button until we have progressed to that screen