Speeding up data import to Neo4j v5 and CSV format data about hetionet

[dhimmel]

Awesome work @nickzren. Nice job finding an efficient import method that works with the latest Neo4j stack.

I took a quick look at the changes and I'll need a little more time to think about where the code belongs... since it could possibly live in dhimmel/integrate or hetio/hetnetpy rather than in this repo whose focus is more the data and not the code to generate the data.

Taking a step back, there's a couple contributions that will be of major utility (in order of importance/interest):

1. a neo4j dump file that is compatible with neo4j 5 (and possibly future neo4j versions)
2. code to generate the neo4j database and dump file (i.e. your csv branch)
3. the csv files, but I'm a little cautious in that they have some similarities with the TSV files and we'd want to understand and document the differences.

Any thoughts?

[nickzren]

Thanks @dhimmel

I recognize the primary aim of this repo is data-centric rather than code-centric. However, I respectfully suggest that including data-specific scripts, such as Python and Cypher, could add value. No strong opinion here, and I'll defer to your ultimate decision on the matter.

The CSV data is derived from JSON files and serves as a comprehensive dataset, including all properties for both nodes and edges.

CSV files and Cypher scripts are generated simultaneously to ensure data type consistency, making the data import into Neo4j unaffected by version changes.

Having separate CSV files for nodes and edges not only enhances the framework's comprehensibility but also allows users to easily choose or modify or extend the data.

I encountered difficulties with restoring from Neo4j dump files and couldn't resolve the issues, so I began exploring alternative solutions.

I'm still learning about this and graph databases, so any corrections are welcome.