Why not build Entity features after Direct features？ about featuretools

[kmax12]

Hi @zhxt95,

Thanks for the interesting question! The reason for the ordering in Featuretools is to avoid creating duplicate features. If we were to create the direct features before creating the transform features, a primitive could be applied to make same feature in two different ways.

As an example, suppose we have two entities transactions and customers and a feature date of birth in customers. When we use ft.dfs with target entity transactions we would create both customers.TimeSince(date of birth) and TimeSince(customers.date of birth). By doing direct features after transforms, we only create the latter.

This is something that could be enhanced in a future release of dfs in Featuretools.

[zhxt95]

Well, as you said, duplicate features are indeed annoying problem. Yet I still think it is necessary to build direct features first.
For example, suppose there are two entities. Entity customer contains features job and outgoing each week while entity payroll tells us the salary of each job. If we want to know whether a customer's incoming can cover his outgoing, a feature like Subtract(payroll.salary, outgoing) is very useful.
With ft.dfs, now I can not automatically generate a feature like this no matter what depth I set. But with the idea in your paper, I can get what I want.

[zhxt95]

I've noticed that the situation I mentioned above is using a trans_primitives with two input features of which one is a direct feature and another is not.
For this kind of situation, duplicating will not be a problem any more.
So I think build efeat after dfeat and use trans only for this situation may possible be a better way.

[kmax12]

@zhxt95 That example makes sense. We'd be happy to extend ft.dfs to handle that case. Are you interested in attempting to make a PR with the change you're suggesting?

Basically, the PR would swap the order of building transform and direct features, but then add in additional logic to avoid features that would calculate the same thing. The primary challenge here is to make that logic as straightforward and general possible.

If you take a stab at this, we'd be happy to discuss and help!

[zhxt95]

I've made #123 as a solution, but I'm not sure if it works well. I'd appreciate it if you can review it and give some suggestions:)