autotelik / datashift_spree

Comprehensive import and export tools for Spree and Spree Products

DataShift Spree

Import and Export Spree E-Commerce models through .xls or CSV files, including all associations.

Create and assign taxons, properties, shipping, tax categories and more through single spreadsheet.

Versions

This release has been tested against Spree 3.1

Installation

Requires datashift.

Add to bundle :

gem 'datashift'
gem 'datashift_spree'

Most functionality provided via command line tools, so create a high level .thor file

thor will search root or lib/tasks so for example you can place it in lib/tasks/shop.thor

And then copy in the following

# You may/may not need this next line depending on your Rails/thor setup
require File.expand_path('config/environment.rb')

require 'datashift'
require 'datashift_spree'

DataShift::load_commands
DataShift::SpreeEcom::load_commands

To check the available tasks run thor list with a search term, for example

    bundle exec thor list datashift
    bundle exec thor list datashift_spree

To check latest usage information use thor help <command>

for example

    bundle exec thor help datashift_spree:load:products

Usage

Template Generation

For loading data, probably the first thing you'll want to do is create an Excel template for the model(s) you wish to import. There is a general task datashift:generate:excel ( or datashift:generate:csv ) for generating a template for any model.

There are some higher level tasks, specifically for producing the specific template for loading Spree Products

thor datashift_spree:template:product -t tmp/product_template.xls

To also include all possible associations

thor datashift_spree:template:product -m Spree::Product --associations -t tmp/full_product_template.xls

A large number of example Spreadsheets with headers and comments, can be found in the spec/fixtures directory - including .xls and csv versions for simple Products or complex Products with multiple/nested Taxons, Variants, Properties etc

Excel versions contain column headings with **Comments ** with instructions on supported syntax for each column.

The same DSL syntax is supported in both Excel and CSV versions.

To get detailed information on the impact/usage of each column see the Spree guides

For example to understand the promotionable field see . https://guides.spreecommerce.org/user/promotions/

Data Import/Export

Once you have data prepared you can import it using task :

thor datashift_spree:load:products

  -i, --input=INPUT                            # The import file (.xls or .csv)
  -s, [--sku-prefix=SKU_PREFIX]                # Prefix to add to each SKU before saving Product
  -p, [--image-path-prefix=IMAGE_PATH_PREFIX]  # Prefix to add to image path for importing from disk
  -v, [--verbose], [--no-verbose]              # Verbose logging
  -c, [--config=CONFIG]                        # Configuration file containg defaults or over rides in YAML
  -d, [--dummy], [--no-dummy]                  # Dummy run, do not actually save Image or Product

Dummy Run is very useful to drive out any issues without actually impacting the database. All changes are rolled back.

A summary of the import is printed to the console,and incase of errors the datashift log ('log/datashift.log') will contain full details.

For example, not setting compulsory fields would show up as :

E, [2020-02-13T12:12:31.469743 #24410] ERROR -- : Failure(s) reported :
E, [2020-02-13T12:12:31.469757 #24410] ERROR -- : 	Shipping Category can't be blank
E, [2020-02-13T12:12:31.469762 #24410] ERROR -- : 	Price can't be blank

Simple Product

Variant Prices/SKUs

To assign different SKUs or Prices to each variant, datashift supports two special columns

variant_price
variant_sku

These should contain pipe '|' delimited lists of the prices, or SKUs, to assign, to each Variant available, and should therefor contain exactly the same number of entries as Variants available.

N.B These columns should come AFTER the Variant creation columns, as the Variants must exists at the time these columns are processed.

Example

variant_price	                 variant_sku

171.56|260.44|171.56|260.44	TARR.SFOP424EW0|TARR.SFOP424EW3|TARR.SFOP414EW0|TARR.SFOP414EW3     
119.33|208.23	                MOLE.SFOP140EA0|MOLE.SFOP140EA3
110.00|198.00	                TALL.SFOP140EW0|TALL.SFOP140EW3
54.89|109.78|69.24	        CHET.SFOP128EW3|CHET.SFOP140EW0|CHET.SFOP140EW3
42.22	                        LOST.REDL218EW0

Creating Association data - Excel or CSV

Testing

We use RSpec, so tests located in the spec subdirectory.

To test this gem we require an actual Spree store, so when the specs are first run a dummy Rails app is created containing a Spree store, whose version we can control in spec/Gemfile so it's easy to change the Spree version and re-run the specs.

It's therefor recommended that all testing be done in spec dir itself, so first cd into spec

Edit spec/Gemfile and set the version of Spree you wish to test against and run bundler :

    cd spec
    bundle install

When changing Spree versions, you should force a rebuild of a clean sandbox, and removing the Gemfile.lock will resolve any funny version issues, so run:

    cd spec
    rm -rf dummy
    rm -rf Gemfile.lock

thor datashift:spree_tasks:build_sandbox

The next time you run rspec the sandbox will be regenerated using the latest versions of Rails/Spree specified in your Gemfile

    bundle exec rspec -c .

License

Copyright:: (c) Autotelik Media Ltd 2012

Author :: Tom Statter

Date :: Oct 2012

The MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

datashift_spree's Issues

some required fields for products are not generated in template

I'm working on importing products right and some of the required fields are not generated and put into the template spreadsheet.

datashift_spree/lib/thor/spree/products_images.thor

Line 63 in dc83b5d

opts[:mandatory] = ['sku', 'name', 'price']

#products_images.thor
opts[:mandatory] = ['sku', 'name', 'price'] #only name is generated

Fields generated by
bundle exec thor datashift:generate:excel -m Spree::Product -r spree_product_template.csv --remove-rails

• name
• description available_on
• deleted_at permalink
• meta_description
• meta_keywords
• tax_category_id
• shipping_category_id
• msrp
• short_name
• notes
• supplier_sku

Spree version support

No where in the documentation can I find what version's of spree are supported by this gem?

Would love to know if spree 2.2 is currently supported? Thanks heaps
Luke

How to avoid duplicating?

Running command bundle exec thor datashift_spree:load:products -i path and all is imported to DB perfectly, but with loading again duplicates appear. How can I run the command so that it would update existing products and variants? and added new if they appeared in the file?

WARNING: These headings couldn't be mapped to class Spree::Product : ["SKU", "Images"]

I have a .csv file with this header: SKU,Name,Description,Price,Taxons,Images

When I run "bundle exec thor datashift:import:csv -i products_finished.csv -m Spree::Product" I get the following warning:

WARNING: These headings couldn't be mapped to class Spree::Product : ["SKU", "Images"]

No products are imported. This surprises me since the example .csv and .xls files have SKU and Images columns. What am I doing wrong?

Spree 2.4.3: Count on hand not being loaded with products (.xls)

I'm trying to load some products and everything goes as expected except for the count_on_hand stock items. It remains 0. My column looks liem this:

Variants | StockItem
newborn-clothes-size:0M;color:branco | count_on_hand:1

Spree v2.4.3

What am I missing here?!

Taxons keep failing to import

From all the docs indicate it is just "categories>test"

id,sku,price,map,name,description,available_on,deleted_at,slug,meta_description,meta_keywords,tax_category_id,shipping_category_id,created_at,updated_at,product_option_types,option_types,properties,product_properties,classifications,taxons,variants,variants_including_master,prices,stock_items,line_items,orders,variant_images,promotion_rules
,test3,3.99,5.99,test3,test3,,,test3,test3,test3,1,1,,,,,"name:test_name|presentation:Name","test_name:This is a test",,"categories>test>sub-test",,,,,,,,

My log

Spree Loading initialised with:
{:verbose=>true}
Product load from File [spree_products.csv]
Product load using forced operators: [["images", "variant_price", "variant_sku", "price"]]
Starting load from file [spree_products.csv]
Perform Load Options:
{"input"=>"spree_products.csv", "verbose"=>true, "mandatory"=>["sku", "name", "price"], "force_inclusion"=>["images", "variant_price", "variant_sku", "price"]}
  �[1m�[36m (39.1ms)�[0m  �[1mBEGIN�[0m
Processing default value available_on : 2014-09-25 15:43:42
Processing default value cost_price : 0.0
Processing default value price : 0.0
Processing value: []
Populator assigning data via id
Assignging  => [id] (CAST 2 TYPE  0)
Processing value: [test3]
Populator assigning data via sku
Brute force assignment of value  test3 => [sku]
Processing value: [3.99]
Populator assigning data via price
Brute force assignment of value  3.99 => [price]
Processing value: [5.99]
Populator assigning data via map
Brute force assignment of value  5.99 => [map]
Processing value: [test3]
Populator assigning data via name
Assignging test3 => [name] (CAST 2 TYPE  "test3")
Processing value: [test3]
Populator assigning data via description
Assignging test3 => [description] (CAST 2 TYPE  "test3")
Processing value: [2014-09-25 15:43:42]
Populator assigning data via available_on
Assignging 2014-09-25 15:43:42 => [available_on] (CAST 2 TYPE  2014-09-25 15:43:42 UTC)
Processing value: []
Populator assigning data via deleted_at
Assignging  => [deleted_at] (CAST 2 TYPE  nil)
Processing value: [test3]
Populator assigning data via slug
Assignging test3 => [slug] (CAST 2 TYPE  "test3")
Processing value: [test3]
Populator assigning data via meta_description
Assignging test3 => [meta_description] (CAST 2 TYPE  "test3")
Processing value: [test3]
Populator assigning data via meta_keywords
Assignging test3 => [meta_keywords] (CAST 2 TYPE  "test3")
Processing value: [1]
Populator assigning data via tax_category_id
Assignging 1 => [tax_category_id] (CAST 2 TYPE  1)
Processing value: [1]
Populator assigning data via shipping_category_id
Assignging 1 => [shipping_category_id] (CAST 2 TYPE  1)
Processing value: []
Populator assigning data via created_at
Assignging  => [created_at] (CAST 2 TYPE  nil)
Processing value: []
Populator assigning data via updated_at
Assignging  => [updated_at] (CAST 2 TYPE  nil)
Processing value: []
Trying to find operator class with Parent Namespace Spree
  �[1m�[35mSpree::Product Exists (37.6ms)�[0m  SELECT  1 AS one FROM `spree_products`  WHERE `spree_products`.`slug` = BINARY 'test3' LIMIT 1
  �[1m�[36mSpree::Product Exists (33.8ms)�[0m  �[1mSELECT  1 AS one FROM `spree_products`  WHERE `spree_products`.`slug` = BINARY 'test3' LIMIT 1�[0m
  �[1m�[35mSQL (38.0ms)�[0m  INSERT INTO `spree_products` (`available_on`, `created_at`, `description`, `id`, `meta_description`, `meta_keywords`, `name`, `shipping_category_id`, `slug`, `tax_category_id`, `updated_at`) VALUES ('2014-09-25 15:43:42', '2014-09-25 19:43:43', 'test3', 0, 'test3', 'test3', 'test3', 1, 'test3', 1, '2014-09-25 19:43:43')
  �[1m�[36mSpree::Variant Exists (39.9ms)�[0m  �[1mSELECT  1 AS one FROM `spree_variants`  WHERE `spree_variants`.`sku` = BINARY 'test3' AND `spree_variants`.`deleted_at` IS NULL LIMIT 1�[0m
  �[1m�[35mSQL (37.2ms)�[0m  INSERT INTO `spree_variants` (`cost_currency`, `cost_price`, `map`, `product_id`, `sku`, `updated_at`) VALUES ('USD', 0.0, 5.99, 0, 'test3', '2014-09-25 19:43:43')
  �[1m�[36mSpree::Variant Load (40.1ms)�[0m  �[1mSELECT  `spree_variants`.* FROM `spree_variants`  WHERE `spree_variants`.`id` = 40 LIMIT 1�[0m
  �[1m�[35mSQL (35.1ms)�[0m  INSERT INTO `spree_prices` (`amount`, `currency`, `variant_id`) VALUES (3.99, 'USD', 40)
  �[1m�[36mSQL (37.4ms)�[0m  �[1mUPDATE `spree_variants` SET `spree_variants`.`updated_at` = '2014-09-25 19:43:43' WHERE `spree_variants`.`id` = 40�[0m
  �[1m�[35mSpree::Product Load (44.4ms)�[0m  SELECT  `spree_products`.* FROM `spree_products`  WHERE `spree_products`.`id` = 0 LIMIT 1
  �[1m�[36mSpree::StockLocation Load (39.6ms)�[0m  �[1mSELECT `spree_stock_locations`.* FROM `spree_stock_locations`�[0m
  �[1m�[35mSpree::StockLocation Load (51.9ms)�[0m  SELECT  `spree_stock_locations`.* FROM `spree_stock_locations`  WHERE `spree_stock_locations`.`id` = 1 LIMIT 1
  �[1m�[36mSpree::StockItem Exists (36.0ms)�[0m  �[1mSELECT  1 AS one FROM `spree_stock_items`  WHERE (`spree_stock_items`.`variant_id` = BINARY 40 AND `spree_stock_items`.`stock_location_id` = 1 AND `spree_stock_items`.`deleted_at` IS NULL) LIMIT 1�[0m
  �[1m�[35mSQL (35.8ms)�[0m  INSERT INTO `spree_stock_items` (`backorderable`, `created_at`, `stock_location_id`, `updated_at`, `variant_id`) VALUES (1, '2014-09-25 19:43:44', 1, '2014-09-25 19:43:44', 40)
  �[1m�[36mSQL (43.5ms)�[0m  �[1mUPDATE `spree_variants` SET `spree_variants`.`updated_at` = '2014-09-25 19:43:44' WHERE `spree_variants`.`id` = 40�[0m
  �[1m�[35mSQL (36.2ms)�[0m  UPDATE `spree_products` SET `spree_products`.`updated_at` = '2014-09-25 19:43:44' WHERE `spree_products`.`id` = 0
  �[1m�[36m (41.1ms)�[0m  �[1mSELECT MAX(`spree_variants`.`position`) AS max_id FROM `spree_variants`  WHERE `spree_variants`.`deleted_at` IS NULL AND `spree_variants`.`product_id` = 0 AND `spree_variants`.`is_master` = 0�[0m
  �[1m�[35mSQL (40.5ms)�[0m  UPDATE `spree_variants` SET `spree_variants`.`position` = 1 WHERE `spree_variants`.`id` = 40
  �[1m�[36m (43.5ms)�[0m  �[1mSELECT SUM(`spree_stock_items`.`count_on_hand`) AS sum_id FROM `spree_stock_items` INNER JOIN `spree_stock_locations` ON `spree_stock_locations`.`id` = `spree_stock_items`.`stock_location_id` WHERE `spree_stock_items`.`deleted_at` IS NULL AND `spree_stock_items`.`variant_id` = 40 AND `spree_stock_locations`.`active` = 1�[0m
  �[1m�[35mSQL (37.5ms)�[0m  UPDATE `spree_products` SET `spree_products`.`updated_at` = '2014-09-25 19:43:44' WHERE `spree_products`.`id` = 0
  �[1m�[36mSpree::Variant Exists (41.6ms)�[0m  �[1mSELECT  1 AS one FROM `spree_variants`  WHERE (`spree_variants`.`sku` = BINARY 'test3' AND `spree_variants`.`id` != 40) AND `spree_variants`.`deleted_at` IS NULL LIMIT 1�[0m
  �[1m�[35mSQL (41.2ms)�[0m  UPDATE `spree_variants` SET `is_master` = 1, `updated_at` = '2014-09-25 19:43:44' WHERE `spree_variants`.`id` = 40
  �[1m�[36mSQL (54.8ms)�[0m  �[1mUPDATE `spree_products` SET `spree_products`.`updated_at` = '2014-09-25 19:43:44' WHERE `spree_products`.`id` = 0�[0m
  �[1m�[35mSpree::Taxon Load (78.0ms)�[0m  SELECT `spree_taxons`.* FROM `spree_taxons` INNER JOIN `spree_products_taxons` ON `spree_taxons`.`id` = `spree_products_taxons`.`taxon_id` WHERE `spree_products_taxons`.`product_id` = 0
  �[1m�[36mSQL (39.6ms)�[0m  �[1mUPDATE `spree_taxons` SET `spree_taxons`.`updated_at` = '2014-09-25 19:43:45' WHERE 1=0�[0m
  �[1m�[35mSQL (43.2ms)�[0m  UPDATE `spree_taxonomies` SET `spree_taxonomies`.`updated_at` = '2014-09-25 19:43:45' WHERE 1=0 ORDER BY spree_taxonomies.position
Processing value: []
add_options_variants []
Processing value: [name:test_name|presentation:Name]
Trying to find operator class with Parent Namespace Spree
Parsed inbound data into name << test_name
Scan for multiple has_many associations ["test_name"]
  �[1m�[36mSpree::Property Load (41.2ms)�[0m  �[1mSELECT `spree_properties`.* FROM `spree_properties`  WHERE `spree_properties`.`name` IN ('test_name')�[0m
Scan result #<ActiveRecord::Relation []>
Association properties with key(s) ["test_name"] NOT found - Not added.
Parsed inbound data into presentation << Name
Scan for multiple has_many associations ["Name"]
  �[1m�[35mSpree::Property Load (37.9ms)�[0m  SELECT `spree_properties`.* FROM `spree_properties`  WHERE `spree_properties`.`presentation` IN ('Name')
Scan result #<ActiveRecord::Relation []>
Association properties with key(s) ["Name"] NOT found - Not added.
Processing value: [test_name:This is a test]
Parsed inbound data into test_name << This is a test
  �[1m�[36mSpree::Property Load (43.7ms)�[0m  �[1mSELECT  `spree_properties`.* FROM `spree_properties`  WHERE `spree_properties`.`name` = 'test_name' LIMIT 1�[0m
  �[1m�[35mSQL (38.4ms)�[0m  INSERT INTO `spree_properties` (`created_at`, `name`, `presentation`, `updated_at`) VALUES ('2014-09-25 19:43:45', 'test_name', 'Test name', '2014-09-25 19:43:45')
Created New Property #<Spree::Property id: 20, name: "test_name", presentation: "Test name", created_at: "2014-09-25 19:43:45", updated_at: "2014-09-25 19:43:45">
  �[1m�[36mSQL (36.5ms)�[0m  �[1mINSERT INTO `spree_product_properties` (`created_at`, `property_id`, `updated_at`, `value`) VALUES ('2014-09-25 19:43:45', 20, '2014-09-25 19:43:45', 'This is a test')�[0m
  �[1m�[35mSQL (39.2ms)�[0m  UPDATE `spree_product_properties` SET `product_id` = 0, `updated_at` = '2014-09-25 19:43:45' WHERE `spree_product_properties`.`id` = 75
  �[1m�[36mSQL (33.1ms)�[0m  �[1mUPDATE `spree_products` SET `spree_products`.`updated_at` = '2014-09-25 19:43:45' WHERE `spree_products`.`id` = 0�[0m
Created New ProductProperty #<Spree::ProductProperty id: 75, value: "This is a test", product_id: 0, property_id: 20, created_at: "2014-09-25 19:43:45", updated_at: "2014-09-25 19:43:45", position: 0>
Processing value: []
Trying to find operator class with Parent Namespace Spree
Processing value: [categories>test>sub-test]
Failed to process row [0] ([nil, "test3", "3.99", "5.99", "test3", "test3", nil, nil, "test3", "test3", "test3", "1", "1", nil, nil, nil, nil, "name:test_name|presentation:Name", "test_name:This is a test", nil, "categories>test>sub-test", nil, nil, nil, nil, nil, nil, nil, nil])
  �[1m�[35m (47.2ms)�[0m  COMMIT

The Product, Variant and Properties are working, but I cannot get Taxons working.

master price is reset if using variants

The master price is reset to 0 if I am using multiple variants. In product_loader.rb, I check @load_object.price before and after variants are saved.

              if(@load_object.variants.size == values.size)
                @load_object.variants.each_with_index {|v, i| v.price = values[i].to_f }
binding.pry
                @load_object.save
binding.pry
              else
                puts "WARNING: Price entries did not match number of Variants - None Set"
              end

However, if I move the price column in the spreadsheet way over to the right, it will avoid the variants save/reset of price.

1.3 findings

Hello,

This is my first time using data shift, and I'm wanting to migrate products from a spree 1.2.2 site to test a new 1.3 site. Please excuse any basic misunderstandings :)

I've installed a spree 1.3 site and run both a datashift:generate:excel and datashift:export:excel on Spree:Product

Neither of these files contained a sku or a price column, so attempting to import the empty exported file generates the error:

ERROR: Mandatory column missing - expected column 'sku'
ERROR: Mandatory column missing - expected column 'price'

When I manually add the columns to the generated and exported file and re-import I get:

Loading from CSV file: export.xls
Processing 1 rows
/Users/jupiter/.rvm/gems/ruby-1.9.3-p327@spree13/gems/datashift-0.11.1/lib/datashift/method_detail.rb:147:in is_a?': class or module required (TypeError) from /Users/jupiter/.rvm/gems/ruby-1.9.3-p327@spree13/gems/datashift-0.11.1/lib/datashift/method_detail.rb:147:inassign'
from /Users/jupiter/.rvm/gems/ruby-1.9.3-p327@spree13/gems/datashift-0.11.1/lib/loaders/loader_base.rb:381:in process' from /Users/jupiter/.rvm/gems/ruby-1.9.3-p327@spree13/gems/datashift_spree-0.3.0/lib/loaders/spree/product_loader.rb:109:inprocess'
from /Users/jupiter/.rvm/gems/ruby-1.9.3-p327@spree13/gems/datashift-0.11.1/lib/loaders/csv_loader.rb:67:in `block (3 levels) in perform_csv_load'
etc...

Am I missing something simple here? Or will this plugin need updating to support 1.3?

Thanks for the great work. This is one of those essential plugins so I really appreciate the work you've done on it to date.

Thanks for your time.

Cheers,

Jupiter

Error while executing thor datashift_spree:load:products

I've an error while running:
thor datashift_spree:load:products --input=product.csv

Error trace:

/usr/local/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require': cannot load such file -- spree (LoadError)
    from /usr/local/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
    from /usr/local/lib/ruby/gems/2.0.0/gems/datashift_spree-0.5.0/lib/helpers/spree_helper.rb:24:in `<top (required)>'
    from /usr/local/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
    from /usr/local/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
    from /usr/local/lib/ruby/gems/2.0.0/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:17:in `load_thorfile'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/util.rb:155:in `class_eval'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/util.rb:155:in `load_thorfile'
    from /usr/local/lib/ruby/gems/2.0.0/gems/datashift_spree-0.5.0/lib/datashift_spree.rb:117:in `block in load_commands'
    from /usr/local/lib/ruby/gems/2.0.0/gems/datashift_spree-0.5.0/lib/datashift_spree.rb:115:in `each'
    from /usr/local/lib/ruby/gems/2.0.0/gems/datashift_spree-0.5.0/lib/datashift_spree.rb:115:in `load_commands'
    from /PATH_TO_RAILS_APP/es.thor:5:in `load_thorfile'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/util.rb:155:in `class_eval'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/util.rb:155:in `load_thorfile'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/runner.rb:214:in `block in initialize_thorfiles'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/runner.rb:213:in `each'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/runner.rb:213:in `initialize_thorfiles'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/runner.rb:32:in `method_missing'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:29:in `run'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:126:in `run'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /usr/local/lib/ruby/gems/2.0.0/gems/thor-0.19.1/bin/thor:6:in `<top (required)>'
    from /usr/local/bin/thor:23:in `load'
    from /usr/local/bin/thor:23:in `<main>'

Any help?

Thanks

Is it possible to import translated products ?

If yes, could you help me to find the good syntax in the CSV or XLS ?

Thank you !

undefined method `get_spree_class'

Thanks for update! I am now using that.

The below

bundle exec thor datashift:spree:images -i /home/jet/AceLeatherGoods/Ace_Primary_Web_Photos/ -s

...is now generating this error:

datashift_spree-d5ba969c8c64/lib/thor/spree/products_images.thor:128:in images': undefined methodget_spree_class' for DataShift::SpreeHelper:Module (NoMethodError)

Spree::Product : ["count_on_hand"] couldn't be mapped

in production:
bundle exec thor datashift_spree:load:products -i import.csv
DataShift::Product starting upload from file: import.csv
WARNING: These headings couldn't be mapped to class Spree::Product :
["count_on_hand"]

in development is although not working.

in csv file is count_on_hand: 100
the development.log shows this insert in mysql database:

INSERT INTO spree_stock_items (backorderable, count_on_hand, created_at, stock_location_id, updated_at, variant_id) VALUES (1, 0, '2013-06-06 04:57:09', 1, '2013-06-06 04:57:09', 90)

why is count_on_hand: 0?

my csv file:

sku,name,description,meta_description,meta_keywords,price,variants,product_properties,taxons,count_on_hand,images
sku_product_0001,Product Name 0001,Description 0001,description for google,keywords for google,399.99,colour:blue;size:small | colour:red;size:medium | colour:green;size:small ,color:green | color:black,Paintings|Water Colour>Color,100,import_images/rails.png|import_images/rails1.png

thanks Michael

No Thor commands available

I am using spree 2.4, rails (4.1.11), jruby 1.7.19, java version "1.7.0_95".

bundle exec thor list
WARNING: unable to load thorfile "/root/my_store20151008/mighty.thor": uninitialized constant DataShift::Logging
org/jruby/RubyModule.java:2726:in `const_missing'
No Thor commands available

Errno::EMFILE: Too many open files

I'm trying to load products through
thor datashift_spree:load:products

It uploads 30 products correctly but with last 7 I receive this errors:

Errno::EMFILE: Too many open files - identify -format %wx%h '/var/folders/fv/2h1c2j0166j__vsmpthby25c0000gn/T/stream20130316-8844-1jqp5ji.jpg[0]'>

ERROR: Failed to process image from URL http://...

Add products images filenames to export csv

My problem is that i'm not sure is this is already supported by default on the gem but when i export all my spree products to a csv, or excel, file it won't add also the product image filename.

I'm using the command:
bundle exec thor datashift:export:excel -m Spree::Product -r spree_product_export.xls

Is this feature already supported and is it impossible to also add this value to the data?

change count_on_hand in templates to stock_items

Needs to be updated in XLS templates

Heroku integration

Incredible project! I'm adding some additional command line options and adding support for Spree 2. Happy to provide a PR when i'm done.

My question is: do you have any ideas how to run this on Heroku? I'll give it a shot, but wanted to see if you'd thought about it first.

-matt

Price Can't Be Blank

Hi There,

Having trouble with a CSV import:

Cannot Save - Invalid Spree::Product Record - ["Price can't be blank", "Shipping category can't be blank"]

My CSV definitely has a price field, in fact if I change it to something like master_price it complains it's missing the price field

Here's the 1st couple of lines of my cdv

sku,name,description,available_on,properties,price,cost_price,shipping_category,

EK00056,The Time Is Now - Moloko,The Time Is Now - Moloko,2014-10-01,downloadable: Yes,02.00,0.50,default,

how to upload order and users

there is rake task to upload users and orders?

Update existing rows

Is it not possible to update existing rows?

Fore instance this works

"sku","name","description","price","cost_price","map","shipping_category_id","tax_category_id"
"SIMPLE_003","Simple third row avail in future","more blah blah",50.34,23.34,300,1,1

but running the same thing again with a price change

"sku","name","description","price","cost_price","map","shipping_category_id","tax_category_id"
"SIMPLE_003","Simple third row avail in future","more blah blah",60.34,23.34,300,1,1

fails. It creates a duplicate row in spree_products

~/Documents/sites/rvw(branch:develop*) » bundle exec thor datashift_spree:load:products -i products.csv
DataShift::Product starting upload from file: products.csv
["sku", "name", "price"]
["sku", "name", "description", "price", "cost_price", "map", "shipping_category_id", "tax_category_id"]



Loading from CSV file: products.csv
Processing 1 rows
DEBUG: insistent_assignment : sku= => SIMPLE_003 (String)
DEBUG: insistent_assignment : price= => 50.34 (String)
DEBUG: insistent_assignment : cost_price= => 23.34 (String)
DEBUG: insistent_assignment : map= => 300 (String)

Processing Summary Report
>>>>>>>>>>>>>>>>>>>>>>>>>
Processed total of 1 entry
1 objects were succesfully processed.
There were NO failures.
------------------------------------------------------------
~/Documents/sites/rvw(branch:develop*) » bundle exec thor datashift_spree:load:products -i products.csv
DataShift::Product starting upload from file: products.csv
["sku", "name", "price"]
["sku", "name", "description", "price", "cost_price", "map", "shipping_category_id", "tax_category_id"]



Loading from CSV file: products.csv
Processing 1 rows
DEBUG: insistent_assignment : sku= => SIMPLE_003 (String)
DEBUG: insistent_assignment : price= => 60.34 (String)
DEBUG: insistent_assignment : cost_price= => 23.34 (String)
DEBUG: insistent_assignment : map= => 300 (String)

Processing Summary Report
>>>>>>>>>>>>>>>>>>>>>>>>>
Processed total of 1 entry
0 objects were succesfully processed.
WARNING : Check logs : 1 rows contained errors and 1 records NOT created.

Spree verison 3.0.4

Hi, I have run into several issues with Spree version 3.0.4

Is the gem currently compatible with this version?

Thanks

ERROR: "thor csv" was called with arguments ["-i,", "-m,"] Usage: "thor datashift:import:csv -i, --input=INPUT -m, --model=MODEL"

What is this? No docs? What's the right syntax please?

Uninitialized constant DataShift::SpreeEcom::ImageLoader::ModelMethodsManager (NameError)

Since today I can't run : bundle exec thor datashift_spree:load:attach_images

Sample command :

bundle exec thor datashift_spree:load:attach_images -i db/products/csv/images.csv

I'm running the latest version with Spree 2.4 and Rails 4.1.8

From GEMFILE

gem 'datashift', github: 'autotelik/datashift'
gem 'datashift_spree', github: 'autotelik/datashift_spree', :branch => 'master'

From GEMFILE.lock

remote: git://github.com/autotelik/datashift_spree.git
revision: 37f79a7
branch: master
specs:
datashift_spree (0.6.0)
datashift (> 0.15, >= 0.15.0)
mechanize (> 2.6, >= 2.6.0)

The problem is visible on a clean install of Spree with nothing but datashift and default Spree 2.4 config

I remember using it about a month ago without any issues.

Any idea?
Stacktrace :
/Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/bundler/gems/datashift_spree-37f79a7bf671/lib/loaders/spree/image_loader.rb:28:in initialize': uninitialized constant DataShift::SpreeEcom::ImageLoader::ModelMethodsManager (NameError) from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/bundler/gems/datashift_spree-37f79a7bf671/lib/thor/spree/products_images.thor:83:innew'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/bundler/gems/datashift_spree-37f79a7bf671/lib/thor/spree/products_images.thor:83:in attach_images' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/command.rb:27:inrun'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor.rb:359:indispatch'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/base.rb:440:in start' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/runner.rb:36:inmethod_missing'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/command.rb:29:in run' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/command.rb:126:inrun'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor.rb:359:indispatch'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/lib/thor/base.rb:440:in start' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/gems/thor-0.19.1/bin/thor:6:in<top (required)>'
from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/bin/thor:23:in load' from /Users/lawebshopimac/.rvm/gems/ruby-2.1.2@mchoc/bin/thor:23:in

'

Could not find command "datashift_spree:products".

When I run "bundle exec thor help datashift_spree:products" I get this error message: Could not find command "datashift_spree:products".

I also get this error when trying to import products using the -i option. I checked to make sure that my gems for datashift and datashift_spree are installed. What am I doing wrong?

Export doesn't work

git:(datashift_spree) ✗ bundle exec thor datashift:export:csv -m Spree::Product -r spree_product_export.csv
/Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/datashift-43b33fb51833/lib/thor/export.thor:85:in `csv': uninitialized constant Thor::Sandbox::Datashift::Export::ModelMapper (NameError)
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/command.rb:27:in `run'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor.rb:359:in `dispatch'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/base.rb:440:in `start'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/runner.rb:36:in `method_missing'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/command.rb:29:in `run'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/command.rb:126:in `run'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor.rb:359:in `dispatch'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/lib/thor/base.rb:440:in `start'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bundler/gems/thor-0bb59c16f2b8/bin/thor:6:in `<top (required)>'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bin/thor:23:in `load'
    from /Users/bci20/Sites/equ-001-website-2015/.gems/bin/thor:23:in `<main>'

1.2 Compatible?

Are there any plans for supporting spree version 1.2? Anyone test it??
Great extension. Congrats.

Feature: spree_related_products support

Hi! First of all thank you so much for your wonderful work! It's super useful. It would be great to support spree_related_products plugin as well. Is it possible?

Cheers!

uninitialized constant DataShift::CsvLoading (NameError)

gems/datashift_spree-0.5.0/lib/loaders/spree/spree_base_loader.rb:19:in `class:SpreeBaseLoader': uninitialized constant DataShift::CsvLoading (NameError)

Stack Trace --

from personal/.rvm/gems/ruby-2.3.0@project/gems/datashift_spree-0.5.0/lib/loaders/spree/spree_base_loader.rb:18:in <module:DataShift>' from personal/.rvm/gems/ruby-2.3.0@project/gems/datashift_spree-0.5.0/lib/loaders/spree/spree_base_loader.rb:16:in <top (required)>'
from personal/.rvm/gems/ruby-2.3.0@project/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require' from personal/.rvm/gems/ruby-2.3.0@project/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'
from personal/.rvm/gems/ruby-2.3.0@project/gems/datashift_spree-0.5.0/lib/loaders/spree/product_loader.rb:8:in <top (required)>' from personal/.rvm/gems/ruby-2.3.0@project/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'
from personal/.rvm/gems/ruby-2.3.0@project/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require' from personal/.rvm/gems/ruby-2.3.0@project/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:42:in products'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/command.rb:27:in run' from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/base.rb:440:in start'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/runner.rb:36:in method_missing' from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/command.rb:29:in run'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/command.rb:126:in run' from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/lib/thor/base.rb:440:in start'
from personal/.rvm/gems/ruby-2.3.0@project/gems/thor-0.19.1/bin/thor:6:in <top (required)>' from personal/.rvm/gems/ruby-2.3.0@project/bin/thor:23:in load'
from personal/.rvm/gems/ruby-2.3.0@project/bin/thor:23:in <main>' from personal/.rvm/gems/ruby-2.3.0@project/bin/ruby_executable_hooks:15:in eval'
from personal/.rvm/gems/ruby-2.3.0@project/bin/ruby_executable_hooks:15:in `

'

Fails to import previously exported model

1. Do export of Spree's Product model (output format does not matter)
   bundle exec thor datashift:export:csv -m Spree::Product -r products_out.csv
2. Try to import it back
   bundle exec thor datashift:import:csv -i products_out.csv -m Spree::Product

Loading from CSV file: products_out.csv
Processing 16 rows

Processing Summary Report

Processed total of 16 entries
0 objects were succesfully processed.
WARNING : Check logs : 16 rows contained errors and 16 records NOT created.
CSV loading stage complete - 0 rows added.

So it fails to import the same previously exported data.
Log file does not describe the issue, just 'Failed to process row [nn]' without any additional info.
Passing `-v' parameter at import command instead of turning on verbosity fails with

vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.14/lib/active_support/tagged_logging.rb:72:in method_missing': undefined methodverbose' for #ActiveSupport::BufferedLogger:0x0000000345cc40 (NoMethodError)

Seems like an another bug.

#<NoMethodError: undefined method `stock_location_name=' for #<Spree::Product:0x00000007ecd9a0>>

This error appears in logfile.

I think this explains why it is not working count on hands. Any ideas how to fix it, gentlemen?

Begin processing Row 1 from CSV file
Applying default value [available_on] on (available_on)
Populator assign - [2016-03-03 15:55:25 -0700] via available_on (assignment)
Assign 2016-03-03 15:55:25 -0700 => [available_on](CAST 2 TYPE 2016-03-03 22:55:25 UTC)
Applying default value [CostPrice] on (cost_price)
Populator assign - [0] via cost_price (assignment)
Brute force assignment of value 0 => [cost_price]
No operator found for default [stock_location_name] trying basic assignment
#<NoMethodError: undefined methodstock_location_name=' for #Spree::Product:0x00000007ecd9a0>
`
Failed to assign [default] via operator stock_location_name
Badly specified default - could not set stock_location_name(default)
Applying default value [shipping_category] on (shipping_category)
Populator assign - [default] via shipping_category (belongs_to)

Spree 2.4
datashift - master
datashift_spree - master

Failed to import supplier_id (spree_drop_ship)

Hi
We are using "spree-contrib/spree_drop_ship" extension.
This extension allows us to assign products to different suppliers.

We are able to upload products using datashift excel, but all uploaded products get assigned to "Spree Admin" user.

So, we mentioned "supplier_id" column within datashift excel, to assign products to specific "supplier", when we try to import this excel, this fails.

Please cloud you suggest the right way to assign "spree products" to specific "spree suppliers" ..

Thank you for your help!!

Uninitialized constant DataShift:Logging

Hi Tom,

with new install using jRuby 1.7.0.preview and Spree 1.1.2, getting following error when

bundle exec thor list datashift_spree

WARNING: unable to load thorfile "/home/jet/.rvm/gems/jruby-1.7.0.preview1@Spree_1.1.2/bundler/gems/datashift_spree-51d917967aff/lib/thor/spree/bootstrap_cleanup.thor": uninitialized constant DataShift::Logging

Jet

thor datashift_spree:load:products fails to render values from xls formulas

Was using a function in the .xls file, but it was not rendering the value correctly.

.xls function
=CONCATENATE("EAN:", U2)

incorrect value rendered

#<Spreadsheet::Formula:0x0000000b0ca908

here's a fix in populator.rb. Add another elsif to check for Spreadsheet::Formula ancestor.

        if( value.is_a? ActiveRecord::Relation ) # Rails 4 - query no longer returns an array
          @current_value = value.to_a
        elsif( value.class.ancestors.include?(Spreadsheet::Formula))
          @current_value = value.value
        elsif( value.class.ancestors.include?(ActiveRecord::Base) || value.is_a?(Array))
...

About multiple variants

Hello.
For our current project we're collecting some information about products from one web store and each product from that store has many variants (about 5-10), so we thought the way that you suggest in wiki in which we should place info about all variants of one product in one single line in csv file is quite inconvenience, so we decided to try to add to csv file variants of one product line by line (for example: first line is a base product, second line is a variant of this product), but, as we could expect - it didn't work (got this error Spree::Product#master_images delegated to master.images, but master is nil and many others), so, if you can help us - please help.
P.S This is an example of what we tried to do

Product assigned to every parent taxonomy

Not sure if it was intended design, or not. If I specify a taxonomy in the Taxons column, the product is added to every parent taxonomy instead of only the child.

E.g, product is added to
Health & Beauty > Personal Care > Cosmetics > Cosmetic Tools > Skin Care Tools > Foot Files
Health & Beauty > Personal Care > Cosmetics > Cosmetic Tools > Skin Care Tools
Health & Beauty > Personal Care > Cosmetics > Cosmetic Tools
... etc

can't import product

I did generate spree product template and export spree product. But could not import spree product.
I used this command line:
bundle exec thor datashift:import:excel -i --input=spree_product.xls -m --model=Spree::Product -a

response is No value provided for option '--input'

uninitialized constant DataShift::SpreeHelper::SpreeBaseLoader (NameError)

In my Gemfile:
gem 'datashift', github: 'autotelik/datashift'
gem 'datashift_spree', github: 'autotelik/datashift_spree'

When running
bundle exec thor datashift_spree:load:attach_images --input=/vagrant/imageimport/import.xls

I receive the error:
/var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:19:in <module:SpreeHelper>': uninitialized constant DataShift::SpreeHelper::SpreeBaseLoader (NameError) from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:10:inmodule:DataShift'
from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:8:in <top (required)>' from /var/lib/gems/1.9.1/gems/polyglot-0.3.5/lib/polyglot.rb:65:inrequire'
from /var/lib/gems/1.9.1/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require' from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:77:inattach_images'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:27:in run' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:ininvoke_command'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/base.rb:440:instart'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/runner.rb:36:in method_missing' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:29:inrun'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:126:in run' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:ininvoke_command'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/base.rb:440:instart'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/bin/thor:6:in <top (required)>' from /usr/local/bin/thor:23:inload'
from /usr/local/bin/thor:23:in `

'

attach_images `initialize': wrong number of arguments (2 for 1) error

Hi again @autotelik I've just updated to the latest gem and when I try to run the attach_images with the following command:

bundle exec thor datashift:spree:attach_images -i image_load.csv

I get:

Users/dev/.rvm/gems/ruby-1.9.2-p320@spree/gems/datashift_spree-0.2.1/lib/loaders/spree/image_loader.rb:32:in `initialize': wrong number of arguments (2 for 1) (ArgumentError)

I've looked into it a bit more and I think it might be line 78 in thor/spree/product_images.thor:

loader = DataShift::SpreeHelper::ImageLoader.new(nil, options)

Where the image_loader initialize is expecting a single argument:

def initialize(options = {})

I'm setup to fork and do a pull request, but am I thinking down the right lines, or have I missed something?

can't attach images on product update

gemfile:
ruby '2.3.0'
gem 'rails', '4.2.5'
gem 'spree', '~> 3.1.0'
gem 'datashift', github: 'autotelik/datashift'
gem 'datashift_spree', github: 'autotelik/datashift_spree', :branch => 'feature/update_api_latest_ds_and_spree'

I can't attach images during Product creation. I've added the image field and attempted with both url for the file or full path. Running thor datashift_spree:load:products -i filepath.xls does create the product but the image is now.

When instead trying to add the image to existing products with thor datashift_spree:load:attach_images -i filepath.xls I get the following argument error complaint. The spreadsheet uses a sku and an attachment

bundler/gems/datashift_spree-8fb9c3ae017a/lib/loaders/spree/image_loader.rb:26:in `initialize': wrong number of arguments (given 2, expected 0) (ArgumentError)

thor datashift:paperclip:attach -a Spree::Image -k Spree::Variant -f images --attach-to-find-by-field=sku -i filepath.xls --split_file_name_on _
returns

WARNING : 1 of 1 files could not be attached to a Spree::Image For your convenience copying files with MISSING Spree::Variant to : MissingAttachmentRecords Created 0 / 1 attachments of type Spree::Image attached to Spree::Variant
with this log output

D, [2017-09-13T23:57:01.101313 #21725] DEBUG -- : Catalogue - building operators information for Spree::Variant
I, [2017-09-13T23:57:01.117192 #21725]  INFO -- : Loading attachments from vendor/datashift/templates/product_images.xls
I, [2017-09-13T23:57:01.145786 #21725]  INFO -- : Loading objects of type Spree::Image
I, [2017-09-13T23:57:01.145842 #21725]  INFO -- : Found 1 files - splitting names on delimiter [_]
I, [2017-09-13T23:57:01.145863 #21725]  INFO -- : Finding matching field/association [images] on class [Spree::Variant]
D, [2017-09-13T23:57:01.146041 #21725] DEBUG -- : Column [images] (0) - mapped to :
      Class         [Spree::Variant]
      Operator Type [has_many]
      Operator      [images]

I, [2017-09-13T23:57:01.146065 #21725]  INFO -- : Processing attachment file product_images.xls 
I, [2017-09-13T23:57:01.146091 #21725]  INFO -- : Attempting to find attachment owner (Spree::Variant for [product_images]
I, [2017-09-13T23:57:01.146107 #21725]  INFO -- : Scanning for record where Spree::Variant.sku ~=  product_images
D, [2017-09-13T23:57:01.147343 #21725] DEBUG -- :   �[1m�[35mSpree::Variant Load (0.2ms)�[0m  SELECT  "spree_variants".* FROM "spree_variants" WHERE "spree_variants"."deleted_at" IS NULL AND (lower(sku) = 'product_images')  ORDER BY "spree_variants"."id" ASC LIMIT 1
I, [2017-09-13T23:57:01.147466 #21725]  INFO -- : Nothing found - trying split file_name to terms on [_]
I, [2017-09-13T23:57:01.147486 #21725]  INFO -- : Scanning by term for record where sku ~=  product
D, [2017-09-13T23:57:01.147903 #21725] DEBUG -- :   �[1m�[36mSpree::Variant Load (0.1ms)�[0m  �[1mSELECT  "spree_variants".* FROM "spree_variants" WHERE "spree_variants"."deleted_at" IS NULL AND (lower(sku) = 'product')  ORDER BY "spree_variants"."id" ASC LIMIT 1�[0m
I, [2017-09-13T23:57:01.147974 #21725]  INFO -- : Scanning by term for record where sku ~=  images
D, [2017-09-13T23:57:01.148372 #21725] DEBUG -- :   �[1m�[35mSpree::Variant Load (0.1ms)�[0m  SELECT  "spree_variants".* FROM "spree_variants" WHERE "spree_variants"."deleted_at" IS NULL AND (lower(sku) = 'images')  ORDER BY "spree_variants"."id" ASC LIMIT 1
D, [2017-09-13T23:57:01.148815 #21725] DEBUG -- :   �[1m�[36mSpree::Variant Load (0.1ms)�[0m  �[1mSELECT  "spree_variants".* FROM "spree_variants" WHERE "spree_variants"."deleted_at" IS NULL AND (lower(sku) = '_product')  ORDER BY "spree_variants"."id" ASC LIMIT 1�[0m
D, [2017-09-13T23:57:01.149262 #21725] DEBUG -- :   �[1m�[35mSpree::Variant Load (0.1ms)�[0m  SELECT  "spree_variants".* FROM "spree_variants" WHERE "spree_variants"."deleted_at" IS NULL AND (lower(sku) = 'product_images')  ORDER BY "spree_variants"."id" ASC LIMIT 1
E, [2017-09-13T23:57:01.149351 #21725] ERROR -- : Failure(s) reported :
E, [2017-09-13T23:57:01.149369 #21725] ERROR -- : 	No matching owner found for file name : product_images
I, [2017-09-13T23:57:01.149417 #21725]  INFO -- : Copying vendor/datashift/templates/product_images.xls to MissingAttachmentRecords folder

Importing of Images no longer works with latest Spree

After updating to Spree 2.0.3 loading products with an Images column no longer works. There is no obvious reason for the failure in the logs. The images are created in the spree_assets table but creation of the product fails.

import multiple images with whitespace between pipe not working

Hi,

this works: import_images/rails.png|import_images/rails1.png

this not: import_images/rails.png | import_images/rails1.png

thanks

Products price get a trailing zero in import

When I import products that only have a master variant I get a trailing zero added to the price and cost price.
123 => 1230

I'm using SpreeProducts.xls from fixtures. I'm using Spree 3.0. The store's currency is ISK.

Question: Is it possible to create/update existing variants and products?

My use case is to import a spreadsheet with products and their master variants (also which variant becomes master), updating any existing records in the database.

Update
I've been getting this error but figured out this is because one of my column headers is empty.

*** NoMethodError Exception: undefined method `operator' for nil:NilClass

(rdb:1) puts e.backtrace.join("\n")
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/loader_base.rb:288:in `prepare_data'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:7in `each_with_index'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:74:in `block (2 levels) in perform_csv_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:55:in `each'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:55:in `each_with_index'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:55:in `block in perform_csv_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/activerecord-3.2.13/lib/active_record/transactions.rb:208:in `transaction'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/csv_loader.rb:52:in `perform_csv_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift-0.13.0/lib/loaders/loader_base.rb:118:in `perform_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift_spree-0.5.0/lib/loaders/spree/spree_base_loader.rb:49:in `perform_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift_spree-0.5.0/lib/loaders/spree/product_loader.rb:46:in `perform_load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:65:in `products'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/command.rb:27:in `run'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/invocation.rb:120:in `invoke_command'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor.rb:363:in `dispatch'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/base.rb:439:in `start'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/runner.rb:36:in `method_missing'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/command.rb:29:in `run'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/command.rb:128:in `run'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/invocation.rb:120:in `invoke_command'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor.rb:363:in `dispatch'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/lib/thor/base.rb:439:in `start'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/gems/thor-0.18.1/bin/thor:6:in `<top (required)>'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/bin/thor:19:in `load'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/bin/thor:19:in `<main>'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/bin/ruby_noexec_wrapper:14:in `eval'
/Users/<user>/.rvm/gems/ruby-1.9.3-p429@<namespace>/bin/ruby_noexec_wrapper:14:in `<main>'
nil'

Config YAML Explanation in README.md

It would be great if we could get some kind of explanation of how to use the Config YAML file.

Also, it would be helpful to get an explanation of what fields are required and why. Ultimately, how we can map a CSV provided to us by a supplier with a Config YAML so that we can create clean imports.

If someone could steer me in the right direction, I'd be happy to write it. Feel free to reply here or msg me directly on Github.

Thanks,

E

Add variant_sku and price

I have added to csv fields as it is indicated https://github.com/autotelik/datashift_spree/wiki/Spree-Import-Export-Tips#variants
variant_sku . before saving
https://github.com/autotelik/datashift/blob/master/lib/loaders/loader_base.rb#L346
I output @load_object.variants and it shows me variants with filled sku as in csv but after saving they are changed to standard ones.

Request: import taxons

I ran into a need to import a few hundred taxons in preparation for manual product entry work. I ended up importing a single fake product that belonged to all of the taxons that I wanted to create but it would be much cleaner if there was an import routine that removed the need for the fake product.

Ideally the taxon import would use the same taxon syntax as the product import. It would probably be sufficient for the taxon import to only require a taxon field and not worry about the other taxon record elements.

Dan

Paperclip not working for spree 3?

Is this function working correctly?

I cannot seem to get it working. The "f" option gives me a name error.

bundle exec thor datashift:paperclip:attach -a Spree::Image -k Spree::Variant -f sku --attach-to-find-by-field sku -i /home/ubuntu/workspace/public/spree/images/

Using Field sku for lookup
/usr/local/rvm/gems/ruby-2.3.0/gems/datashift-0.16.0/lib/thor/paperclip.thor:105:in `attach': uninitialized constant Thor::Sandbox::Datashift::Paperclip::ModelMapper (NameError)

#<Errno::EMFILE: Too many open files - convert '/tmp/stream20130526-5632-1lvcvad.gif' -coalesce -resize "600x600>" -strip -auto-orient '/tmp/stream20130526-5632-1lvcvad20130526-5632-bg9rvm'>

I get the above error message when importing my products. I have 11k+ products that I am trying to import, most of them have images. Do you have any idea on where to start debugging this? I am more than willing to help.

undefined method `get_spree_class'

Duplicate

Error to upload images using datashift_spree:load:images

datashift_spree : 0.5.0

Was trying to upload images using sku. got following error

/home/vishal/.rvm/gems/ruby-2.2.4/gems/datashift-0.16.0/lib/loaders/paperclip/attachment_loader.rb:53:in `initialize': wrong number of arguments (4 for 1..3) (ArgumentError)
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:146:in `new'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:146:in `images'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/runner.rb:36:in `method_missing'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/command.rb:29:in `run'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/command.rb:126:in `run'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /home/vishal/.rvm/gems/ruby-2.2.4/gems/thor-0.19.1/bin/thor:6:in `<top (required)>'
    from /home/vishal/.rvm/gems/ruby-2.2.4/bin/thor:23:in `load'
    from /home/vishal/.rvm/gems/ruby-2.2.4/bin/thor:23:in `<main>'
    from /home/vishal/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `eval'
    from /home/vishal/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `<main>'

Wired thing is that lib/loaders/paperclip/attachment_loader.rb:53in 'initialize' method the have 3rd parameter as hash, then also throwing error.