problem with finetuning about protein_bert

[r-kellerm]

@kattens - is it possible that you code isn't up to date? This is already fixed here.

[kattens]

I'm not so sure where the problem is since I am basically only trying to run the exact same example in the notebook in colab. but I am still seeing the same errors.

[nadavbra]

@kattens Did you try to install the new version of ProteinBERT as @r-kellerm suggested?

[kattens]

I am following the instructions for installation:

!pip install protein-bert

then I got this:

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting protein-bert
Downloading protein_bert-1.0.1-py3-none-any.whl (39 kB)
Requirement already satisfied: tensorflow in /usr/local/lib/python3.10/dist-packages (from protein-bert) (2.9.0)
Collecting tensorflow-addons (from protein-bert)
Downloading tensorflow_addons-0.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (591 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 591.0/591.0 kB 11.0 MB/s eta 0:00:00
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from protein-bert) (1.22.4)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from protein-bert) (1.5.3)
Requirement already satisfied: h5py in /usr/local/lib/python3.10/dist-packages (from protein-bert) (3.8.0)
Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from protein-bert) (4.9.2)
Collecting pyfaidx (from protein-bert)
Downloading pyfaidx-0.7.2.1-py3-none-any.whl (28 kB)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->protein-bert) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->protein-bert) (2022.7.1)
Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from pyfaidx->protein-bert) (1.16.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from pyfaidx->protein-bert) (67.7.2)
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.6.3)
Requirement already satisfied: flatbuffers<2,>=1.12 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.12)
Requirement already satisfied: gast<=0.4.0,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (0.4.0)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (0.2.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.54.0)
Requirement already satisfied: keras<2.10.0,>=2.9.0rc0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (2.9.0)
Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.1.2)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (16.0.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (3.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (23.1)
Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (3.19.6)
Requirement already satisfied: tensorboard<2.10,>=2.9 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (2.9.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (0.32.0)
Requirement already satisfied: tensorflow-estimator<2.10.0,>=2.9.0rc0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (2.9.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (2.3.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (4.5.0)
Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow->protein-bert) (1.14.1)
Collecting typeguard<3.0.0,>=2.7 (from tensorflow-addons->protein-bert)
Downloading typeguard-2.13.3-py3-none-any.whl (17 kB)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow->protein-bert) (0.40.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2.17.3)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (3.4.3)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2.27.1)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (1.8.1)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2.3.0)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (5.3.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (1.3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2022.12.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (3.4)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (2.1.2)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (0.5.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.10,>=2.9->tensorflow->protein-bert) (3.2.2)
Installing collected packages: typeguard, pyfaidx, tensorflow-addons, protein-bert
Successfully installed protein-bert-1.0.1 pyfaidx-0.7.2.1 tensorflow-addons-0.20.0 typeguard-2.13.3

I am not quite sure if this is not the updated version.

[nadavbra]

Can you try to install directly from the latest GitHub repo code (rather than pip):

git clone ...
git submodule init
git submodule update
python setup.py install

[kattens]

I have tried that too.
but still, this error showed up:
[2023_06_16-15:15:26] Training set: Filtered out 0 of 14945 (0.0%) records of lengths exceeding 510.
[2023_06_16-15:15:27] Validation set: Filtered out 0 of 1661 (0.0%) records of lengths exceeding 510.
[2023_06_16-15:15:27] Training with frozen pretrained layers...
Epoch 1/5
468/468 [==============================] - ETA: 0s - loss: 0.0959

WARNING:tensorflow:evaluate() received a value for sample_weight, but weighted_metrics were not provided. Did you mean to pass metrics to weighted_metrics in compile()? If this is intentional you can pass weighted_metrics=[] to compile() in order to silence this warning.

468/468 [==============================] - 1551s 3s/step - loss: 0.0959 - val_loss: 0.0984 - lr: 0.0100
Epoch 2/5
468/468 [==============================] - ETA: 0s - loss: 0.0729

WARNING:tensorflow:evaluate() received a value for sample_weight, but weighted_metrics were not provided. Did you mean to pass metrics to weighted_metrics in compile()? If this is intentional you can pass weighted_metrics=[] to compile() in order to silence this warning.

468/468 [==============================] - 1449s 3s/step - loss: 0.0729 - val_loss: 0.0797 - lr: 0.0100
Epoch 3/5
468/468 [==============================] - ETA: 0s - loss: 0.0717

WARNING:tensorflow:evaluate() received a value for sample_weight, but weighted_metrics were not provided. Did you mean to pass metrics to weighted_metrics in compile()? If this is intentional you can pass weighted_metrics=[] to compile() in order to silence this warning.

468/468 [==============================] - 1347s 3s/step - loss: 0.0717 - val_loss: 0.0691 - lr: 0.0100
Epoch 4/5
468/468 [==============================] - ETA: 0s - loss: 0.0708

WARNING:tensorflow:evaluate() received a value for sample_weight, but weighted_metrics were not provided. Did you mean to pass metrics to weighted_metrics in compile()? If this is intentional you can pass weighted_metrics=[] to compile() in order to silence this warning.

Epoch 4: ReduceLROnPlateau reducing learning rate to 0.0024999999441206455.
468/468 [==============================] - 1386s 3s/step - loss: 0.0708 - val_loss: 0.0705 - lr: 0.0100
[2023_06_16-16:51:08] Training the entire fine-tuned model...
[2023_06_16-16:51:35] Incompatible number of optimizer weights - will not initialize them.

[nadavbra]

What error?

[kattens]

Epoch 4: ReduceLROnPlateau reducing learning rate to 0.0024999999441206455.
468/468 [==============================] - 1386s 3s/step - loss: 0.0708 - val_loss: 0.0705 - lr: 0.0100
[2023_06_16-16:51:08] Training the entire fine-tuned model...
[2023_06_16-16:51:35] Incompatible number of optimizer weights - will not initialize them.
Epoch 1/5
305/468 [==================>...........] - ETA: 23:33 - loss: 0.0669

[nadavbra]

It looks fine.

[kattens]

But it's not. It's in a loop that doesn't let the model train and finish. This loop continues and never ends.

[nadavbra]

What kind of loop are you talking about? Looks like the model is just training for the number of epochs it's supposed to.

[nadavbra]

@Chinjuj2017 I see you already posted this issue elsewhere. Please avoid duplicate issues in the future.

[Chinjuj2017]

I am sorrry about that, i will remove the duplicate issue. Regards,

…

On Wed, 13 Sept 2023, 1:58 pm Nadav Brandes, ***@***.***> wrote: Closed #56 <#56> as completed. — Reply to this email directly, view it on GitHub <#56 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIBQGMHXKRQ4K6G3DTUUQYDX2FVBVANCNFSM6AAAAAAZGT2GNE> . You are receiving this because you were mentioned.Message ID: ***@***.***>