Enable initial set of node traversals about chainverse-frontend

[Azharo]

Some thoughts on traversals involving more relationships than we can display -- https://www.figma.com/file/kmLdrsPeXs9BZqupjDmfpS/Big-Data-Traversals?node-id=2%3A37

Here is the query for this one. It returns each of the variables as a list.

Curators - Wallets that have created a block referencing the Entity
Members - Wallets that are direct members of Entity from DaoHaus or have voted on a proposal in Snapshot. Probably need to think of creating a :IS_MEMBER relationship for Wallets and Snapshot Space because we would want to keep an updated list of members based on if they still have the token or not right?
Proposals - Right now we don't have "titles" for DaoHaus proposals so need to bring that in on the backend so proposals are only Snapshot ones
Blocks - self-explanatory
Tags - the tags within blocks referencing the entity
Related Daos - This is where I think we do need some analytics and create this as a derived dataset. Otherwise if we have to try to do this on the fly we risk running into memory issues and/or queries that take too long @clemp I am going to look into just creating some sort of Entity-Entity relationship that we can create in just cypher for now since the only real thing we have at the moment with enough data relating Entities is their members list. Thoughts? For now this will just pull Entities that are referenced in the same blocks

MATCH (n:Entity) WHERE n.name='Diamond DAO'
OPTIONAL MATCH (n)<-[:REFERENCES]-(b:Block)<-[:CREATED]-(curator:Wallet)
OPTIONAL MATCH (n)<-[:IS_MEMBER]-(:Member)<-[:IS_MEMBER]-(m1:Wallet)
OPTIONAL MATCH (n)-[:HAS_PROPOSAL]->(p1:Proposal:Snapshot)<-[:VOTED]-(m2:Wallet)
OPTIONAL MATCH (b)-[:REFERENCES]->(e1:Entity) WHERE e1<>n
OPTIONAL MATCH (b)-[:IS_TAGGED]->(t:Tag)
OPTIONAL MATCH (n)-[:HAS_PROPOSAL]->(p2:Proposal:DaoHaus)
RETURN apoc.coll.union(COLLECT(m1.address),COLLECT(m2.address)) as members, COLLECT(DISTINCT(p2.id)) as proposals, COLLECT(e1.name) as related_entities, COLLECT(t.tag) as tags, COLLECT(curator.address) as curators, COLLECT(b.text) as blocks

[Azharo]

That should be fine

[clemp]

@musubipapi mentioned today on a brainstorming call with @Azharo @angelaliu22 and myself that there are currently technical limitations to how different types of nodes can be presented in the front-end (based on how the GraphQL schema is implemented to call the Neo4j data elements). I don't understand clearly what those limitations are, but wanted to make note that this is something to look into and help solve as this work progresses.

[clemp]

Adding additional context...

This is related to #42 ..

Currently in the GraphQL API called by the front-end service, the type definition for the entity is specific to DAOhaus DAO entities..

chainverse-frontend/src/services/Apollo/typedefs.ts

Lines 12 to 27 in cf064b8

	type Entity {
	"""
	TODO: Should this be restricted
	"""
	uuid: ID @id(autogenerate: true)
	name: String @unique #under assumption name for entities are unique
	id: String @unique
	minScore: Float
	network: Float
	onlyMembers: String
	symbol: String
	address: String @unique
	avatar: String
	about: String
	proposals: [Proposal] @relationship(type: "HAS_PROPOSAL", direction: OUT)
	}

However, in the Chainverse database Entity nodes can represent DAOs of different types, with different properties.

This impacts the search and results that are displayed. So we need a solution to organize our type definitions to handle nodes that are of the same "concept" but differ in properties because of the platform generating that data.

[clemp]

@Azharo check out this Loom video created by @angelaliu22 - about halfway through there's a great mockup of what the result of a traversal would look like: https://www.loom.com/share/4296187943304a57a329e2ad219efd8c

With the information that's currently in chainverse-dev right now you could get started on Cypher queries that would produce these results

[liminal-dog]

Some thoughts on traversals involving more relationships than we can display -- https://www.figma.com/file/kmLdrsPeXs9BZqupjDmfpS/Big-Data-Traversals?node-id=2%3A37

[clemp]

Figma document here: https://www.figma.com/file/jIw9hHIlT5jL7W6Z0M8rgl/Traversal-Brainstorm?node-id=0%3A1

First pass requirements
Focus on entities as first case

• Selected entity from result
• Blocks referencing selected entity
• Other entities the blocks reference
• Member who wrote this block
• Entities this member is a member of (or related to)

[Azharo]

MATCH (d:Entity) WHERE d.name CONTAINS 'JournoDAO'
OPTIONAL MATCH p1=(d)-[]-(block:Block)-[]-(user:Wallet)
OPTIONAL MATCH p2=(user)-[:IS_MEMBER]-(member:Member)-[:IS_MEMBER]-(:Entity)
OPTIONAL MATCH p3=(user)-[:VOTED]-(:Proposal)-[:HAS_PROPOSAL]-(:Snapshot:Entity)
OPTIONAL MATCH p4=(block)-[:REFERENCES]-(e:Entity) WHERE d<>e
RETURN p1,p2,p3,p4
Right now this will get what's needed for first pass but will look somewhat empty for a lot of things

[clemp]

Nice. So I propose we implement this incrementally, first with Note:Block, Tag, Entity nodes because of the additional ontology questions for Members and Related DAOs. Proposal I recommend we either focus on only Snapshot and include those in the first pass of this traversal implementation. or separately solve the DAOhaus identifier issue and bring in Proposal nodes once that's done.

The reason I'm recommending this simple, incremental approach is that it gives us focus on the details of GraphQL query, displaying the information once it's returned, serving that to the interface, etc. Then once we have a baseline it's much simpler to add new data to the query result.

@Azharo anything in this approach you would disagree with?

[clemp]

Reviewing the other GraphQL queries (https://github.com/DiamondDAO/chainverse-frontend/blob/develop/src/services/Apollo/Queries.ts) and Type Definitions (https://github.com/DiamondDAO/chainverse-frontend/blob/develop/src/services/Apollo/typedefs.ts) do you have any ideas on a first pass of the GraphQL query to match the Cypher above?

Ideally we'd have a good process / system for testing and developing these kind of queries locally (working on this here #41). Unless you have a good solution for that, maybe just a quick pass at GraphQL that @musubipapi can test the implementation of?

[clemp]

MATCH (n:Entity) WHERE n.name='Diamond DAO'
OPTIONAL MATCH (n)<-[:REFERENCES]-(b:Block)<-[:CREATED]-(curator:Wallet)
OPTIONAL MATCH (n)<-[:IS_MEMBER]-(:Member)<-[:IS_MEMBER]-(m1:Wallet)
OPTIONAL MATCH (n)-[:HAS_PROPOSAL]->(p1:Proposal:Snapshot)<-[:VOTED]-(m2:Wallet)
OPTIONAL MATCH (b)-[:REFERENCES]->(e1:Entity) WHERE e1<>n
OPTIONAL MATCH (b)-[:IS_TAGGED]->(t:Tag)
OPTIONAL MATCH (n)-[:HAS_PROPOSAL]->(p2:Proposal:DaoHaus)

There are a few discrepancies between the GraphQL API, the underlying Neo4j data model, and the integrity of the data in our database right now that make this more challenging than I originally thought.

Example 1
OPTIONAL MATCH (n)<-[:IS_MEMBER]-(:Member)<-[:IS_MEMBER]-(m1:Wallet)

Our API currently does not have a Member node type. I'm adding code to a branch now that provides a Wallet-[:IS_MEMBER]->[:Entity] relation.

chainverse-frontend/src/services/Apollo/typedefs.ts

Line 27 in 9597696

members: [Wallet] @relationship(type: "IS_MEMBER", direction: IN)

chainverse-frontend/src/services/Apollo/typedefs.ts

Line 54 in 9597696

entities: [Entity] @relationship(type: "IS_MEMBER", direction: OUT)

However, this still does reconcile with the underlying data model and therefore we cannot query. We have to either make changes to our GraphQL API, or make changes to the Neo4j ontology to get this to work. I do not want a solution where we start injecting Cypher queries into the GraphQL queries. This is a kind of hacky solution that hard codes dependency on Neo4j which I don't want.

That said, we can make progress by limiting our query for now to the nodes and relations that aren't complicated. For example:

query Notes {
  notes (where: {entities: {name: "Diamond"}}) { # need to configure so only notes with a Diamond DAO entity get returned.
    text
    entities {
      uuid
      name
    }
    wallet {
      address
    }
  }
}

Which returns data in this format:

{
  "data": {
    "notes": [
      {
        "text": "@Diamond DAO could create more awareness around their research and work, creating distribution opportunities for the talent network. (example: invest in being seen as a important and reliable source for DAO research in the space)\n\n\n\nThis could drive more interest and also more value to the token, once awareness could lead to more people joining the network, at the same time as bringing in more deals.\n",
        "entities": [
          {
            "uuid": "c7dd5a23-9eba-4f62-b1fe-3fc53fa1a989",
            "name": "Diamond"
          }
        ],
        "wallet": {
          "address": "0xF812EFF052fA956FFc43032922C0252040A67308"
        }
      }
    ]
  }
}

So, data cleanup and backend design to do before this is actually functional. But we can start playing with data in a sandbox to show what a traversal would look like.

[Azharo]

I am working on making changes to the GQL api so it has more of our existing data model in there.

However, this still does reconcile with the underlying data model and therefore we cannot query. We have to either make changes to our GraphQL API, or make changes to the Neo4j ontology to get this to work. I do not want a solution where we start injecting Cypher queries into the GraphQL queries. This is a kind of hacky solution that hard codes dependency on Neo4j which I don't want.

The above example isn't that complicated so we can just extend the GQL API via a Query to add it in. However, we are going to have to have certain queries be cypher declarations within GraphQL. Otherwise you are then going to force the database and data model to conform to anything we would ever need from an application layer. You'll start to build an umanagable database and then data governance becomes a nightmare as this grows.

I've been meaning to just create a direct relationship between Wallets and Entities because it is something we need anyways. This is going to become more of a data/engineering strategy question, is it the knowledge graph or the API that is the main feature for Chainverse? If the API then we are going to become somewhat like The Graph

[Azharo]

So the above requirements are satisfied by the following GQL query

query {
    snapshotEntities (where: {
      name: "SporkDAO"
      })
      {
      
      blocks {
        ... on Note {
          text
          tags {
            tag
          }
          wallet {
            address
          }
        }
        ... on Response {
          text
          tags {
            tag
          }
          wallet {
            address
          }
        }
      }
      proposals {
        votes {
          address
        }
        title
      }
    }
    daoHausEntities (where: {
      name: "SporkDAO"
      })
      {
      blocks {
        ... on Note {
          text
          tags {
            tag
          }
          wallet {
            address
          }
        }
        ... on Response {
          text
          tags {
            tag
          }
          wallet {
            address
          }
        }
      }
      members {
        address
      }
      proposals {
        id
      }
    }
}```

If the GQL Explorer wasn't a thing this would be a lot harder but with the explorer creating this query is pretty straightforward just clicking buttons and the complexity is really now just on creating a proper set of type definitions.
Unfortunately we have to call it for each Entity type because GQL forces us to use fragments. On the client site we would have to aggregate this.
Other solution is to create a single DaoEntity type that consolidates Spaces and DaoHaus but then we'd also have to do the same in the db and that just creates a lot of complexity.