alteryx / featuretools_sql Goto Github PK

• https://conda-forge.org/docs/maintainer/adding_pkgs.html

• https://github.com/alteryx/featuretools/blob/main/.github/workflows/lint_check.yaml

I should be able to use the select_only parameter when calling get_entityset, not just when calling populate_dataframes

Suppose user has 100 tables, and wants to include all but n. It may be useful to have a do_not_select parameter instead of having to enumerate the 100-n tables they do want in the select_only argument.

• https://github.com/alteryx/featuretools/blob/main/.github/workflows/release.yaml

• https://github.com/alteryx/featuretools/blob/main/.github/workflows/unit_tests_with_latest_deps.yaml

• questdb is an open source time-series database
• We should just be able to support this with no or minimal effort (as it is a Postgresql database)
• We may need to update our docs to specify this

• Here is a good example
• https://github.com/alteryx/featuretools/blob/main/setup.cfg

Write blog post introducting featuretools-sql library

• We should verify that mixed-case column names are correctly handled when importing dataframes and relationships.

We should support converting the SQL tables into Dask DataFrames; we can use the functions detailed below:

https://docs.dask.org/en/stable/dataframe-sql.html

• Add a filter parameter that the user can use to specify which tables they want to import from. Update the test cases to add tests that cover this case.

• https://github.com/alteryx/featuretools/blob/main/.github/workflows/unit_tests_with_minimum_deps.yaml

• https://github.com/alteryx/featuretools/blob/main/.github/workflows/minimum_dependency_checker.yaml

• Add MySQL tests that spin up MySQL instance containing featuretools sample data, and allow user to run tests on that

• All project requirements should be specified in pyproject.toml (including dev, test)

• Standard Python practice is to use underscore for folder names

• influxdb is open source time series database
• This open source library has a python client that we can use to support creating an EntitySet
  • https://github.com/influxdata/influxdb-client-python

• These requirements should not be there

complete =
    tensorflow >= 1.14.0; sys_platform!="darwin" or platform_machine!='arm64'
    tensorflow-metal >= 0.4.0; sys_platform=="darwin" and platform_machine=='arm64'
    tensorflow-macos >= 2.8.0; sys_platform=="darwin" and platform_machine=='arm64'
    tensorflow_hub >= 0.4.0

After releasing on conda-forge (#6) , we should update Featuretools's install.md to reflect the availability of featuretools_sql on conda-forge

We may be able to achieve significant performance boosts by replacing read_sql_query with DuckDB's API. For more info, refer here: https://duckdb.org/2021/05/14/sql-on-pandas.html

-Add relationship information for retail dataset to Testing Postgres Database (Retail Demo for all)
-Test that number of output dataframes match input dataframes
-Test types match (this might fail because WW types aren't the same)
-Pull out EntitySet setup and tear down into a reproduceable mechanism that we can reuse in other tests
-Ensure relationship information matches

• Suppose the user attempts to import tables where there is no primary key constraint explicitly defined. In this event, the code that creates the relationships data structure will fail. What is the best behavior in this scenario? For now, we can throw an informational warning and leave it to the user to set a primary key.

• We can use this library to fake a Snowflake instance (rather than using a live instance for testing)
• https://github.com/tekumara/fakesnow

Pandas's IO library is throwing warnings when we use it with a non-SQLAlchemy connector object. Furthermore, there seems to be issues with snowflake.connector objects handling particular data types.

We should decide how we want to resolve this issue, either by working around it or switching to SQLAlchemy.

• The readme should have the following
  • badges
  • a library image
  • example usage
  • benefits of using the library
  • links to documentation

• When new dependencies are released we should test our package with those latest release
• https://github.com/alteryx/featuretools/blob/main/.github/workflows/latest_dependency_checker.yaml

• timescaledb is an open-source time-series SQL database
• We should just be able to support this with no or minimal effort (as it is a Postgresql database)
• We may need to update our docs to specify this

• Need to release again to get Windows compatibility