GithubHelp home page GithubHelp logo

EAV support about masquerade HOT 7 CLOSED

peterjaap avatar peterjaap commented on June 16, 2024
EAV support

from masquerade.

Comments (7)

peterjaap avatar peterjaap commented on June 16, 2024 1

Here's how another tool does it; https://github.com/mpchadwick/dbanon#magento-2

from masquerade.

erikhansen avatar erikhansen commented on June 16, 2024

As a workaround for now, I am running a manual query to remove the one sensitive EAV attribute I found (added by a custom extension):

UPDATE customer_entity_varchar
    SET `value` = "(000) 000-0000"
    WHERE attribute_id =
    (
        SELECT attribute_id FROM eav_attribute
            WHERE attribute_code = "telephone"
            AND entity_type_id = (
                SELECT entity_type_id
                FROM eav_entity_type
                WHERE entity_type_code = "customer"
            )
    );

I found this attribute by reviewing the attribute codes returned from this query:

-- Find all attribute codes that are stored in the EAV tables and aren't category-related
SELECT `eav_attribute`.`attribute_id`,
	`eav_entity_type`.`entity_type_code`,
	`eav_attribute`.`attribute_code`,
	`eav_attribute`.`backend_type`
FROM `eav_attribute` 
LEFT JOIN `eav_entity_type` ON `eav_attribute`.`entity_type_id` = `eav_entity_type`.`entity_type_id`
WHERE 
	`eav_attribute`.`backend_type` != 'static' 
	AND `eav_entity_type`.`entity_type_code` NOT IN ('catalog_product', 'catalog_category')
ORDER BY `eav_entity_type`.`entity_type_code`, `eav_attribute`.`attribute_code`;

On a native Magento Open Source 2.3.2 install (with sample data), the above query won't return any data. However custom extensions frequently add EAV attributes, so you'll want to review this list to ensure data added by custom extensions are properly anonymized.

The above query does return a few attributes on a Magento 2.2.x Commerce installation. For example:

attribute_id entity_type_code attribute_code backend_type
180 customer reward_update_notification int
181 customer reward_warning_notification int
178 order reward_points_balance_refunded integer
179 order reward_salesrule_points integer
191 rma_item condition int
192 rma_item reason int
193 rma_item reason_other varchar
190 rma_item resolution int

from masquerade.

johnorourke avatar johnorourke commented on June 16, 2024

Just considering this... looking at the two linked approaches:

  • both the Divante and dbanon tools have built in logic to parse the Magento EAV structure
  • other systems have different structures (Drupal for example has something similar to EAV)

It seems to be there's a more general solution we could use, for example:

  • on each table, allow a custom query to select the records (eg. select * from catalog_product_entity_varchar where attribute_id=57)
  • allow queries which return a list of the primary keys to work on (eg. select value_id from catalog_product_entity_decimal inner join eav_attribute using (attribute_id) where eav_attribute.attribute_code like '%price%')
  • allow "query providers", classes which can return a SQL query or even return an Illuminate query object - you could have a class for Magento EAV, a class for Wordpress post metadata, a class for Drupal field values, etc.

The 'query provider' idea could even return query objects for other tables - eg. you supply 'catalog_product_entity' and ask it to anonymise the 'cost_price' field, and it could return a query object for the 'catalog_product_entity_decimal' filtered to only records for that attribute.

catalog:
  catalog_product_entity:
    provider:
      - name: magento_eav
    columns:
      ...etc...

OR

catalog:
  custom_table:
    provider:
      - name: custom_query
        query: 'select id from custom_table where field1 = "value1"'

from masquerade.

johnorourke avatar johnorourke commented on June 16, 2024

Thinking further on this, it would be nice to set locale differently for different sets of values, too:

  • Magento example: you might want to set the Faker locale based on the default locale of the website which a customer is attached to
  • Drupal example: you might want to set the Faker locale based on the 'language' field in the EAV-style table of field values

from masquerade.

johnorourke avatar johnorourke commented on June 16, 2024

Working for me now with Magento2. I've also drafted an example of another table provider (not tested yet) - for Wordpress 'meta' tables: getjohn@5581edb

from masquerade.

peterjaap avatar peterjaap commented on June 16, 2024

Thanks to @johnorourke we now have EAV support! 🎉 🥳

from masquerade.

johnorourke avatar johnorourke commented on June 16, 2024

@peterjaap awesome, thanks so much for testing and merging. Glad I could help :)

from masquerade.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.