peptoneltd / dspp-keras Goto Github PK

Hi.
when downloading the datasets, the dspp.py module raises the following error in python3.
TypeError: the JSON object must be str, not 'bytes'

The main reason explains here.

The fast solution is provided here as the first answer, which works for both Py2 and Py3.

Something seems to be quite off. dspp_cnn.py is producing negative CHI2.

5120/6054 [========================>.....] - ETA: 0s - loss: 0.0362 - rmsd: 0.3122 - chi2: -23331.6329 R(l/v_l)=0.87
6054/6054 [==============================] - 1s - loss: 0.0366 - rmsd: 0.3147 - chi2: -24054.9850 - val_loss: 0.0420 - val_rmsd: 0.3410 - val_chi2: -39990.6675

We simply need to have an ability to restart training. I could envision that as an extension of dspp_utils.py. We could use built-in Keras Checkpoint callbacks.

1. Better intro (adapted from the abstract)
2. Propensity (figure)
3. Correct reference
4. Usage license

We will need a decent decoder to plot the results of recurrent networks with embedding.

This will be a major "selling point" for people running scalable, multi GPU training.

Hi,
As I understood from your README.md the propensity values should be between -1 and 1, which is not fulfilled in your dataset when I use dspp-keras package.

Could you please explain what is the issue here?

I put my example here.

x,y=dspp.load_data()
y[0].max()

which results in 2.5.

Could you please explain how you normalize your dataset?
by min-max scaler? normal-distribution scaler?

We have a potential show stopper here. It seems the propensity values are running in -2.5 to 1.2 ranges. This is very worrisome and obviously messes up training.

An obsolete API calls are made in the multi-GPU code (dspp_utils.py). It would be good to make the Merge layers compatible with the latest Keras API.

For safety reasons we should parse JSON as opposed to cPickle. It has been suggested to me by Keras developers.

I have updated dspp-keras with

pip instal dspp-keras --upgrade

All looks good. However, the example produces:

Using TensorFlow backend.
Downloading data from https://github.com/PeptoneInc/dspp-data/blob/master/database.pkl?raw=true
Traceback (most recent call last):
  File "dspp_cnn.py", line 130, in <module>
    X, Y = dspp.load_data()
  File "/usr/local/lib/python2.7/dist-packages/dsppkeras/datasets/dspp.py", line 15, in load_data
    path = get_file(path, origin='https://github.com/PeptoneInc/dspp-data/blob/master/database.pkl?raw=true')
  File "/usr/local/lib/python2.7/dist-packages/dsppkeras/utils/data_utils.py", line 203, in get_file
    raise Exception(error_msg.format(origin, e.errno, e.reason))
Exception: URL fetch failure on https://github.com/PeptoneInc/dspp-data/blob/master/database.pkl?raw=true: None -- Not Found

We need Apache licence model.

We may need to consider adding h5py dependency to pip package.

Traceback (most recent call last):
  File "dspp_cnn.py", line 148, in <module>
    model.save_weights("model.h5")
  File "/home/kamil/.local/lib/python2.7/site-packages/keras/models.py", line 723, in save_weights
    raise ImportError('`save_weights` requires h5py.')
ImportError: `save_weights` requires h5py.

It seems the download of our database is heavily throttled by Github. Can we send a tar.gz and unpack it in the cache directory?

Hi.
I am curious to know how to get Structural disorder probability from dspp-keras API.
I have no idea where I can find it and how I can load it.

Thanks.

The CHI2 code is reporting suspiciously high values. We should compare the outcome of CHI2 with known power divergence in various tests in Scipy.

800 trong biểu đồ kết quả ở plot.png có ý nghĩa là gì?

Just an idea! Instead of weights.

From: https://keras.io/layers/embeddings/

mask_zero: Whether or not the input value 0 is a special "padding" value that should be masked out. This is useful when using recurrent layers which may take variable length input. If this is True then all subsequent layers in the model need to support masking or an exception will be raised. If mask_zero is set to True, as a consequence, index 0 cannot be used in the vocabulary (input_dim should equal size of vocabulary + 1).

Insofar we are reporting collective CHI2 per whole batch. Consequently, the CHI2 values are pathologically big. We should be able to pass batch size as a parameter of get it from Keras.Callback class and divide the CHI2 by it.