neo4j-partners / hands-on-lab-neo4j-and-vertex-ai Goto Github PK

The gitignore is full of cruft that must have been copied from somewhere. Clean out anything unusused.

Should be keyed to report calendar or quarter but isn't. Need help...

Free trials now allow only one ML pipeline. That means we can't run both the embedding and raw notebooks simultaneously. Not sure what to do about this.

It has a note: "# Primary loop. This could take 30 minutes to an hour." That's not going to work in the labs. Need to come up with an alternative solution.

show metagraph

Vertex AI has a new forecasting feature. Figure out if it makes sense for the lab as our data set is a time series.

Although I can delete characters and manually type characters, the ability to paste is very wonky. I eventually got it to work, but I don't know why it finally allowed me to paste.

old syntax.
CREATE CONSTRAINT IF NOT EXISTS ON (p:Company) ASSERT (p.cusip) IS NODE KEY;
CREATE CONSTRAINT IF NOT EXISTS ON (p:Manager) ASSERT (p.filingManager) IS NODE KEY;
CREATE CONSTRAINT IF NOT EXISTS ON (p:Holding) ASSERT (p.filingManager, p.cusip, p.reportCalendarOrQuarter) IS NODE KEY;

replace it with the new syntax
CREATE CONSTRAINT IF NOT EXISTS FOR (p:Company) REQUIRE (p.cusip) IS NODE KEY;
CREATE CONSTRAINT IF NOT EXISTS FOR (p:Manager) REQUIRE (p.filingManager) IS NODE KEY;
CREATE CONSTRAINT IF NOT EXISTS FOR (p:Holding) REQUIRE (p.filingManager, p.cusip, p.reportCalendarOrQuarter) IS NODE KEY;

Right now we load a day of data and then a year. Walking through that Cypher improvement should probably stay. However given the change in GDS approach, we don't need the two data models any more.

This step gives me an error message that mentions "bucket.location" is deprecated". Did google code change?
I'm not sure if I'm doing the step right or not.

Full error message:

:2: DeprecationWarning: Assignment to 'Bucket.location' is deprecated, as it is only valid before the bucket is created. Instead, pass the location to Bucket.create.
bucket.location=REGION

TransportError Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/google/auth/compute_engine/credentials.py in refresh(self, request)
110 try:
--> 111 self._retrieve_info(request)
112 self.token, self.expiry = _metadata.get_service_account_token(

15 frames
TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7f544792bca0>)

The above exception was the direct cause of the following exception:

RefreshError Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/six.py in raise_from(value, from_value)

RefreshError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7f544792bca0>)

There are various syntax issues caused by the version bump from 4 to 5

I really feel like Project ID and Region are probably already set in the managed Vertex AI Workbench environment. But, I'm not seeing what they are set as. Probably should revisit this.

I was able to delete the auth code.

Right now we're telling people to switch the config to workspace. Once that becomes the product default, that's great. But let's not use a non default configuration in the lab. It's just going to confuse people.

We should probably shrink our dataset. There's no reason we need to load this much data for the lab.

Lab 7 is using a text name rather than the CUSIP as a key.

That may have been true for an old version of Data Importer, but it is now able to set relationship properties.
Suggest you remove that line.

We've wound up with a bunch of files in the neo4j-datasets bucket. Some say things like -v2. We need to delete everything that's not being used and name the files in a way that represents their contents.

Example --- https://github.com/googleapis/python-aiplatform/blob/5fe59a4015882d56c22f9973aff888966dd53a2e/tests/unit/aiplatform/test_automl_tabular_training_jobs.py#L1242

Documentation --- https://cloud.google.com/vertex-ai/docs/datasets/prepare-tabular?&_ga=2.93752285.-1780612517.1647714214#ml-use

See if we can cut this down, maybe by processing just one file or cherry picking some smaller ones.

Perspective was created during an earlier step, before data is loaded. Therefore the Perspective is empty. Students will be confused.
Easiest solution is probably to remove the steps during earlier lab where students connect to Bloom.
2nd solution is to tell students to create/import a new perspective

Something obscure is going on when Bloom is first loaded. In some cases, the node types aren't populated. This seems to occur intermittently. Need to figure out what exactly is going on.

We now have a bunch of notebooks. They're all authing to the same Neo4j database. When we had two it wasn't too painful to enter creds. Now that's pretty awful. We need to setup creds in a way that they cascade through all the notebooks. It might just be a walkthrough of how to make an environmental varibale.

Now that we're not doing a supervised learning component in here, it probably makes sense to just switch entirely to the 10-K. Need to discuss.

Use transactions in load

:auto LOAD CSV WITH HEADERS FROM 'https://storage.googleapis.com/neo4j-datasets/form13/form13-v2.csv' AS row
CALL { MATCH (m:Manager {managerName:row.managerName})
MATCH (c:Company {cusip:row.cusip})
MERGE (m)-[r:OWNS {reportCalendarOrQuarter:date(row.reportCalendarOrQuarter)}]->(c)
SET r.value = toFloat(row.value), r.shares = toInteger(row.shares)
} IN TRANSACTIONS OF 1000 ROWS;

Can't seem to use the new batch prediction feature because it doesn't know about multiregion buckets. As a workaround, could try to create it in one region. Not sure how that works. Asked Henry....

Right now the data model doesn't match across labs. To work around it, we're deleting the data and then reloading it. Make the data model match everywhere and remove the delete statements as they'll no longer be needed.

We need to reintroduce fine tuning, either in parsing or the chatbot. Example code for that exists here: https://github.com/neo4j-partners/neo4j-generative-ai-google-cloud

This has a __MAC_OSX directory in it. Likely there's no repeatable script to generate it. Need to look into this...