aimat-lab / gcnn_keras Goto Github PK

Can you implement this architecture derivate from DMPNN https://github.com/SY575/CMPNN ?
reference here: https://www.ijcai.org/proceedings/2020/0392.pdf

For very complex molecules RDKit can return an error to generate 3D molecule conformer: there is a tool free that may help to solve this issue
http://users.abo.fi/mivainio/balloon/
Is it possible to find a way for very complex molecules to take into account this issue

I found this paper https://arxiv.org/pdf/1906.12192.pdf & code https://github.com/microsoft/tf2-gnn

I think this is more interesting than MAT graph. Of course due to speed, I will say the main objective is to get GNN_FiLM, but RGIN, RGAT, RGCN and GGNN are still interesting to implement but in a second time.

I've just discovered that AttentiveFP PyG code uses "Gat version" of attention while in original code they use "Gatv2 version" strategy. Can you check which one you implement in your code please ?

page 5 from here https://arxiv.org/pdf/2105.14491.pdf

I guess your implementation is not correct:

do you add a bias in the dense ?

Can we know which molecules failed during the dataprocess ?

I want to use Optuna but I don't see a way to change the litterature files to get them Optuna compatible ?

Do you see a way to do an example for ChemProp / DMPNN code please ?

After moving to commit eb5b8d8 got the following error running DMPNN model:

ValueError: Shape must be at least rank 2 but is rank 1 for '{{node model/dmpnnp_pooling_edges_directed/dmpnn_gather_edges_pairs/gather_nodes_ingoing/GatherV2}} = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_INT64, batch_dims=0](model/dmpnnp_pooling_edges_directed/dmpnn_gather_edges_pairs/gather_nodes_ingoing/PartitionedCall, model/dmpnnp_pooling_edges_directed/dmpnn_gather_edges_pairs/gather_nodes_ingoing/GatherV2/indices, model/dmpnnp_pooling_edges_directed/dmpnn_gather_edges_pairs/gather_nodes_ingoing/GatherV2/axis)' with input shapes: [?], [], [] and with computed input tensors: input[2] = <1>.

it works at commit 5fed55c

that would be nice to have the "default model hyper" for regression for:

• GAT
• GATv2
• GraphSAGE
• Megnet
• Schnet
• DimenetPP
• NMPN
• GCN

this model https://arxiv.org/abs/2203.09456 looks interesting https://github.com/CIS-group/MolNet

I wonder if we can modify the Labels in a large saved dataset pickle file ?

Hi,

I have a question: how to set "leaky_relu" instead of relu as MLP activation function ?

that would be interesting to test.

Also how can I introduce layernorm in the MLP ?

Can you add an option to add csv descriptors (Molecules plain vector) to DMPNN ?

similar to this

Cause OCHEM produce those type of molecular descriptors I propose to do the version loading from the csv files containing smiles and targets the rest of "Descriptors".

Would be useful to call them Desc0,Desc1, etc... like in OCHEM and be able to plug them by concatenation to the graph feature before the MLP layer like in the chemprop tool.

thanks

/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/utils/adj.py:67: RuntimeWarning: divide by zero encountered in power
d_ii = np.power(d_row, -0.5).flatten()
/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/utils/adj.py:68: RuntimeWarning: divide by zero encountered in power
d_jj = np.power(d_col, -0.5).flatten()

I see this error coming (I find it in DMPNN)

Hi!

Thanks for providing this library. It helps a great deal in working with graphs in Tensorflow.

I have the following issue: I want to perform distributed training using multiple GPUs (using MirroredStrategy). However, the KGCNN layers seem to give problems here when loading a trained model for testing. An illustration with the exact error is provided below. Although I believe you are aware of this issue (it is also indicated in the 'limitations' section of your paper), do you know how to circumvent this?

Thanks in advance. Cheers, Seger

Do you see a way to replace the keras GRU by the Haste_tf GRU version from here https://github.com/lmnt-com/haste/blob/master/docs/tf/haste_tf.md ?

there are lot of advantage in this package for regularization at cuda level etc...

unfortunately I don't see a way to plug the haste_tf at keras level lmnt-com/haste#33 ?

your help would be appreciate

Traceback (most recent call last):
File "GCN-keras.py", line 213, in
dataloader = NumpyTensorList(*[getattr(dataset, x['name']) for x in hyper['model']['inputs']])
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/io/loader.py", line 34, in init
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/io/loader.py", line 37, in _length_test
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/io/loader.py", line 37, in
TypeError: object of type 'NoneType' has no len()

I use the last version of the code 1.1.2 Master bug got an error.

I don't see this error with GraphSage, GAT, GATv2 models parameter

File "", line 219, in _call_with_frames_removed
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/literature/AttentiveFP.py", line 7, in
from kgcnn.layers.keras import Dense, Dropout, LayerNormalization
ImportError: cannot import name 'LayerNormalization' from 'kgcnn.layers.keras' (/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/layers/keras.py)
any help ?

I try to adapt old code to new version and I have this issue for none native dataset.

(tf) tgg@gvalmu00008 training % python AttFP.py config.cfg
ERROR:root:Module 'kgcnn.utils.learning' is deprecated and will be removed in future versions. Please move to 'kgcnn.training'.
WARNING:root:Module 'kgcnn.selection' will be removed in future versions in favour of 'kgcnn.hyper'.
ERROR:root:Module 'kgcnn.utils.data' is deprecated and will be removed in future versions. Please move to 'kgcnn.data.utils'.
ERROR:kgcnn.mol.convert:Can not import OpenBabel module for conversion.
Load config file: config.cfg
Architecture selected: GIN
My parameters
Loss RMSE
LSTM 16
DENSE 16
PROBA 0
LR start 0.01
Metal device set to: Apple M1 Max

systemMemory: 64.00 GB
maxCacheSize: 24.00 GB

2022-07-11 14:01:58.468538: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-07-11 14:01:58.468690: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: )
before True
{'model': {'name': 'GIN', 'inputs': [{'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True}, {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True}], 'input_embedding': {'node': {'input_dim': 96, 'output_dim': 100}}, 'last_mlp': {'use_bias': True, 'units': [200, 100, 1], 'activation': ['kgcnn>leaky_relu', 'kgcnn>selu', 'linear']}, 'depth': 4, 'dropout': 0.1, 'gin_args': {'units': [100, 100], 'use_bias': True, 'activation': ['relu', 'relu'], 'use_normalization': True, 'normalization_technique': 'batch'}, 'output_embedding': 'graph', 'output_mlp': {'activation': 'linear', 'units': 1}}, 'training': {'fit': {'batch_size': 32, 'epochs': 200, 'validation_freq': 1, 'verbose': 2, 'callbacks': []}, 'compile': {'optimizer': {'class_name': 'Adam', 'config': {'lr': {'class_name': 'ExponentialDecay', 'config': {'initial_learning_rate': 0.001, 'decay_steps': 1600, 'decay_rate': 0.5, 'staircase': False}}}}, 'loss': 'mean_absolute_error'}, 'cross_validation': {'class_name': 'KFold', 'config': {'n_splits': 5, 'random_state': None, 'shuffle': True}}}, 'data': {'dataset': {'class_name': 'MoleculeNetDataset', 'config': {}, 'methods': [{'set_attributes': {}}]}, 'data_unit': 'unit'}, 'info': {'postfix': '', 'kgcnn_version': '2.0.3'}}
WARNING:kgcnn.hyper.hyper:Hyperparameter {'model': ...} changed to {'model': {'config': {...}}}
INFO:kgcnn.hyper.hyper:Adding model class to 'model': {'class_name': make_model}
INFO:kgcnn.hyper.hyper:Adding 'postfix_file' to 'info' category in hyperparameter.
INFO:kgcnn.hyper.hyper:Adding 'multi_target_indices' to 'training' category in hyperparameter.
training
INFO:kgcnn.data.MoleculeNetDataset:Generating molecules and store train.sdf to disk...
[14:01:58] Explicit valence for atom # 6 N, 4, is greater than permitted
WARNING:kgcnn.mol.convert:Failed conversion for smile C1C=CC=C2C3N(=CC=CC=3)CCN2=1
[14:01:58] Explicit valence for atom # 11 O, 3, is greater than permitted
[14:01:58] Explicit valence for atom # 6 N, 4, is greater than permitted
WARNING:kgcnn.mol.convert:Failed conversion for smile C1=CC(C2C=CN(C)=CC=2)=CC=N1C
WARNING:kgcnn.mol.convert:Failed conversion for smile C12=CC=C(N(CC)CC)C=C2O=C2C=C(C=CC2=C1C1=CC=CC=C1C(O)=O)N(CC)CC
INFO:kgcnn.data.MoleculeNetDataset: ... converted molecules 466 from 466
INFO:kgcnn.data.MoleculeNetDataset: ... read molecules 0 from 466
[14:02:00] CTAB version string invalid at line 4
[14:02:01] CTAB version string invalid at line 4
[14:02:01] CTAB version string invalid at line 4

WARNING:kgcnn.data.MoleculeNetDataset:Property node_attributes is not set on any graph.
WARNING:kgcnn.data.MoleculeNetDataset:Can not clean property {'shape': [None, 41], 'name': 'node_attributes', 'dtype': 'float32', 'ragged': True} as it was not assigned to any graph.
WARNING:kgcnn.data.MoleculeNetDataset:Property edge_attributes is not set on any graph.
WARNING:kgcnn.data.MoleculeNetDataset:Can not clean property {'shape': [None, 11], 'name': 'edge_attributes', 'dtype': 'float32', 'ragged': True} as it was not assigned to any graph.
INFO:kgcnn.data.MoleculeNetDataset:No invalid graphs for assigned properties found.
dataset
WARNING:kgcnn.data.MoleculeNetDataset:Property node_attributes is not set on any graph.
Traceback (most recent call last):
File "/Users/tgg/Github/tensorflow-m1-setup/gcnn_keras/training/AttFP.py", line 842, in
Xdata = dataset.tensor(inputs)
File "/Users/tgg/miniforge3/envs/tf/lib/python3.9/site-packages/kgcnn-2.0.3-py3.9.egg/kgcnn/data/base.py", line 370, in tensor
return [self._to_tensor(x, make_copy=make_copy) for x in items]
File "/Users/tgg/miniforge3/envs/tf/lib/python3.9/site-packages/kgcnn-2.0.3-py3.9.egg/kgcnn/data/base.py", line 370, in
return [self._to_tensor(x, make_copy=make_copy) for x in items]
File "/Users/tgg/miniforge3/envs/tf/lib/python3.9/site-packages/kgcnn-2.0.3-py3.9.egg/kgcnn/data/base.py", line 362, in _to_tensor
return ragged_tensor_from_nested_numpy(props)
File "/Users/tgg/miniforge3/envs/tf/lib/python3.9/site-packages/kgcnn-2.0.3-py3.9.egg/kgcnn/data/utils.py", line 156, in ragged_tensor_from_nested_numpy
return tf.RaggedTensor.from_row_lengths(np.concatenate(numpy_list, axis=0),
File "<array_function internals>", line 180, in concatenate
TypeError: dispatcher for array_function did not return an iterable

That would be interesting to add the Edge compatible version of GIN:

https://arxiv.org/abs/1905.12265

it's implemented in PYG & dgl
https://pytorch-geometric.readthedocs.io/en/latest/_modules/torch_geometric/nn/conv/gin_conv.html#GINEConv
https://docs.dgl.ai/en/latest/generated/dgl.nn.pytorch.conv.GINEConv.html

This is a nice idea to parallelize the rdkit process but it's not trivial to be cross-platform compatible
training
INFO:kgcnn: Generating molecules and store train.sdf to disk...
Traceback (most recent call last):
File "/Users/tgg/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/mol/gen/default.py", line 2, in
import rdkit
File "/Users/tgg/rdkit2021/rdkit/rdkit/init.py", line 2, in
from .rdBase import rdkitVersion as version
ImportError: dlopen(/Users/tgg/rdkit2021/rdkit/rdkit/rdBase.so, 2): Library not loaded: @rpath/libboost_python38.dylib
Referenced from: /Users/tgg/rdkit2021/rdkit/rdkit/rdBase.so
Reason: image not found
Traceback (most recent call last):
File "AttFPX-keras.py", line 183, in
dataset.prepare_data(overwrite=True, smiles_column_name="smiles", make_conformers=False)
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/data/moleculenet.py", line 100, in prepare_data
mb = self._smiles_to_mol_list(smiles, add_hydrogen=add_hydrogen, sanitize=True,
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/data/moleculenet.py", line 67, in _smiles_to_mol_list
mg = smile_to_mol(smiles[i:i+1000], self.data_directory,
File "/Users/itetko/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/mol/gen/base.py", line 73, in smile_to_mol
raise ValueError("Batch process returned with error:", return_code)
ValueError: ('Batch process returned with error:', CompletedProcess(args=['python3', '/Users/tgg/anaconda3/lib/python3.8/site-packages/kgcnn-1.1.2-py3.8.egg/kgcnn/mol/gen/default.py', '--file', 'ac079a80-7b72-4d4d-8add-0faeb96456d9.smile', '--nprocs', '12', '--sanitize', 'True', '--add_hydrogen', 'True', '--make_conformers', 'False', '--optimize_conformer', 'True'], returncode=1))

I would suggest to make something like this :

from concurrent.futures import ThreadPoolExecutor
from tqdm import tqdm

# if you have to make a list conversion
smis = [smi for smi in SMILES]


# need to be adapt to your code
def getSmilesToMol(smi):
    try:
        return ...
    except:
        return Null

with ThreadPoolExecutor(max_workers=12) as pool:
    res = list(tqdm(pool.map(getSmiles,smis),total=len(smis)))

We need to find a way to get the parameters but it's less complicated than to run "command line"

guillaume

I try to add the labels in input model do you see a method to do so ?

Can you add the name of the model in the png generated file or add a title ?
it will be easier to compare than.

Unfortunately their code is not available, but it's close to “SchNet" - "GVP-GNN" - "Dimenet":

Can you implement it ?

https://arxiv.org/pdf/2102.03150.pdf

If I understand the paper, this is like this :

S = linear(384,(SiLU(linear(128,Sj))))
W = fcut(linear(384,RBF(20,Rij)))
SR = GVP(S,W) => Split into (3)
Vim = Sum(Vj*SR[0]+SR[2]*Rij/L2(Rij))i
DSim = Sum(SR[1])i

having issue to save AttentiveFP model using callback function:

adding a callback to save best model give me an error:
filepath="Best-weights-my_model-{epoch:03d}-{loss:.4f}.hdf5"
ckp = tf.keras.callbacks.ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')

error:
Epoch 00001: val_loss improved from inf to 1.58387, saving model to Best-weights-my_model-001-3.4439.hdf5
Traceback (most recent call last):
File "train_attentionfp_esol.py", line 149, in
validation_freq=epostep, validation_data=(xtest, ytest_), verbose=2)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1137, in fit
callbacks.on_epoch_end(epoch, epoch_logs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py", line 412, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py", line 1249, in on_epoch_end
self._save_model(epoch=epoch, logs=logs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/callbacks.py", line 1301, in _save_model
self.model.save(filepath, overwrite=True, options=self._options)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1979, in save
signatures, options)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/save.py", line 131, in save_model
model, filepath, overwrite, include_optimizer)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 109, in save_model_to_hdf5
model_metadata = saving_utils.model_metadata(model, include_optimizer)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/saving_utils.py", line 154, in model_metadata
model_config['config'] = model.get_config()
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py", line 598, in get_config
return copy.deepcopy(get_network_config(self))
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py", line 1278, in get_network_config
layer_config = serialize_layer_fn(layer)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 245, in serialize_keras_object
config = instance.get_config()
File "/opt/conda/lib/python3.7/site-packages/kgcnn-1.0.2-py3.7.egg/kgcnn/layers/attention.py", line 471, in get_config
KeyError: 'mrecurrent_regularizer'

can we solve this ?

AttentiveFP cannot work with single atom molecule. The original code does not include Hs. We can remove from the dataset this case to avoid "C" smile case:

and

is there a way to have a method to generate the hyper json file ?

I test Lipop this morning and got an issue in the printer log:

any idea what is wrong ?

I have tf 2.3.1 / py 3.7.10

Can you implement LayerNorm and. BatchNorm compatible with RaggedTensor

Is there a way to read directly my 3D molecules coordinates structure "mol" rdkit object (based on corina or ballon) from a given sdf file and my targets coming from a given csv file together ?

maybe something like this can be used

from rdkit.Chem import PandasTools
fn = 'file.sdf'
df = PandasTools.LoadSDF(fn, embedProps=True, molColName=None)

indeed would be nice to be able to convert pandas df mols into graphs

If I use your code I have an error for a list of column index.

I see that generaly you scale our target this is very nice and a very important process but you must not use the full dataset when you do this scaling but only use the training dataset part. It's important to avoid any data leak in model validation process. can you please modify examples to avoid this please ?

for preloaded data we don't have this issue but for new data that would be nice to have a clean way to know what type of data process we need to put in place based on the architecture.

This also include some model / architecture specification like radius or normalization etc... so all meta data transformation applied to make dynamically "architecture + data preprocess" for a given new dataset.

=> I would suggest to store a "3D" boolean parameter: true / false in data json part for each model.

would be nice to make https://github.com/weihua916/powerful-gnns available in your Keras toolkit.

can you implement the Direct MPNN (bond propagation instead of Atom propagation) ?

https://github.com/sxxgrc/molecule-restaurant/blob/d7dfb1cd35ab7729708369c7509111b9b4385054/prediction_model/models/directed_mpnn.py#L11

https://pubs.acs.org/doi/abs/10.1021/acs.jcim.9b00237

https://github.com/chemprop/chemprop

thanks in advance,

GG

Can you add this architecture ?

https://github.com/PKUterran/HamNet

BR

GG

4 topics:

A. transformers:

• MAT ( https://github.com/ardigen/MAT & https://arxiv.org/abs/2002.08264)
• RMAT (https://github.com/gmum/huggingmolecules & https://arxiv.org/abs/2110.05841)

B. Recursive Graphs:

• RGIN ( https://arxiv.org/pdf/1906.12192.pdf + code https://github.com/microsoft/tf2-gnn & https://arxiv.org/abs/1912.11589 + https://github.com/HKUST-KnowComp/NeuralSubgraphCounting, my favorite will be simplest RGIN * )

C. Some finetune versions of existing codes:

• MolNet ( https://arxiv.org/abs/2203.09456 & https://github.com/CIS-group/MolNet)
• MolGIN (https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9189862)
• CMPNN ( https://github.com/SY575/CMPNN & https://www.ijcai.org/proceedings/2020/0392.pdf)
• GemNet: Universal Directional Graph Neural Networks for Molecules (https://github.com/TUM-DAML/gemnet_tf & https://arxiv.org/abs/2106.08903 & https://chemrxiv.org/engage/chemrxiv/article-details/6246035d3affe4aa143c3848 )

D. 3D Equivariants strategies (SE3 => "Chirality friendly" or E => "Achiral ...") essential to have them:

• E(n) Equivariant GNN https://arxiv.org/abs/2102.09844 & https://github.com/lucidrains/egnn-pytorch & https://github.com/vgsatorras/egnn)
• Equivariant GNN for 3D Macromolecular Structure // Steerable-E3-GNN
• SE(3)-equivariant GNN (https://arxiv.org/abs/2106.02347
• https://pubs.acs.org/doi/10.1021/acs.jctc.1c01021 & https://github.com/rinikerlab/EquivariantMultipoleGNN

E. Unsuppervised Graph:

• https://github.com/chao1224/n_gram_graph https://papers.nips.cc/paper/2019/hash/2f3926f0a9613f3c3cc21d52a3cdb4d9-Abstract.html

=> GemNet is very existing model
(*) I think simpler is better so we must focus on low parameter architecture first!

I see this paper to predict Kovats index of molecule and they tweak the Megnet architecture:
https://arxiv.org/pdf/2012.14798.pdf

maybe it's a better option for Megnet type of models ?

In the model prediction step (I use it in a DQN model so fitting happens later) I get following error:

AttributeError: in user code:

    /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:1478 predict_function  *
        return step_function(self, iterator)
    /opt/conda/lib/python3.7/site-packages/kgcnn/layers/pool/pooling.py:46 call  *
        nod, node_part = dyn_inputs[0].values, dyn_inputs[0].row_splits

    AttributeError: 'Tensor' object has no attribute 'values'

I'm running Python version 3.7, TF Version 2.4.1, kgcnn 1.1.1.
The model code :

    s = 12 #Board size
     n = keras.Input(shape=(None, 2), name='node_input', dtype="float32", ragged=True)
    ei = keras.Input(shape=(None, 2), name='edge_index_input', dtype="int64", ragged=True)
    n_in_out = GatherNodes()([n, ei])
    node_messages = Dense(10, activation='relu')(n_in_out)
    node_updates = PoolingLocalMessages()([n, node_messages, ei])
    n_node_updates = Concatenate(axis=-1)([n, node_updates])
    n_embedd = Dense(s**2)(n_node_updates)
    g_embedd = PoolingNodes()(n_embedd)

    inputs = keras.Input(shape=(s,s,16))#,name = 'The game map'
    f = layers.Flatten()(inputs)   
    h,w= s,s
    f = layers.Dense(w*h,activation = "sigmoid")(f)
    f = layers.Reshape((h,w,-1))(f)
    combined = layers.Concatenate()([f, layers.Reshape((h,w,-1))(g_embedd)])
    #cov_combined = layers.Conv2D(4, 2, activation='relu', data_format='channels_last')(combined)
    #cov_combined = layers.Dense(1)(combined)
    units = layers.Dense(6,activation = "softmax")(combined)#,name = "Units_actions"
    cities = layers.Dense(2,activation = "sigmoid")(combined)#,name = "Cities_actions"
    output = layers.Concatenate()([units,cities])
    model = keras.Model(inputs = [n, ei, inputs], outputs = output)
    model.compile(optimizer= "adam", loss= custom_mean_squared_error ,metrics = ["accuracy"])

example for ESOL model set two outputs. If I change to 1 I got an issue why ?

is there a way to save pickle of the dataset object for a very long dataset ?

Discussed in #18

Can we think to split the dataprocess and model in tests ?

that would make the toolkit more flexible.

I see few cases (marginal but real cases) where you have a salt for example in smiles mixture as input. the issue is that during the data preparation we have a crash cause the "Adj" matrice is not inversible (I guess). Can we put an exception in this case and just set invalid instead of a breaking point ?

Can you add a GIN ESOL example please ?

I will try to check Dimenet++ implementation soon. I have made a PR on the PyG with fixed of the code but don't find time to refactor yet.

Very interesting library for graphs. Can you implement the AttentiveFP with your GAT version ?

here the PyG version I implement : https://github.com/rusty1s/pytorch_geometric/blob/master/torch_geometric/nn/models/attentive_fp.py

thanks in advance,

BR
Guillaume

Messages from:
Internal 3d
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['Cl', 'C', 'Br', 'O', 'N', 'P', 'S']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP3, rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 4, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 2, 1, 3]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None]
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None, '1']

External 3D
INFO:kgcnn.mol.encoder:OneHotEncoder Symbol found ['Cl', 'C', 'O', 'P', 'S', 'N', 'Br']
INFO:kgcnn.mol.encoder:OneHotEncoder Hybridization found [rdkit.Chem.rdchem.HybridizationType.SP3, rdkit.Chem.rdchem.HybridizationType.SP2, rdkit.Chem.rdchem.HybridizationType.SP]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalDegree found [1, 4, 3, 2]
INFO:kgcnn.mol.encoder:OneHotEncoder TotalNumHs found [0, 2, 3, 1]
INFO:kgcnn.mol.encoder:OneHotEncoder CIPCode found [None, 'S', 'R']
INFO:kgcnn.mol.encoder:OneHotEncoder ChiralityPossible found [None, '1']

I've also observed that the CIP did not provide same response with internal / external 3D dataset.

do you know what it's like this ?

is this is the "num_targets": 1 it the right value in other methods we have the output_mlp

could it be possible to implement this code / paper:

• https://github.com/ardigen/MAT. https://arxiv.org/abs/2002.08264
• https://github.com/lucidrains/molecule-attention-transformer (I think this code is clearer and cleaner)