I have followed the tips and use the command proposed, but the result like the f1-score is very low. Below is the training log, does someone know what is the problem ?
INFO Training with
{ 'attention_mode': 'label',
'batch_size': 8,
'best_model_path': None,
'bidirectional': 1,
'd_a': 512,
'dropout': 0.3,
'embedding_file': 'data/embeddings/word2vec_sg0_100.model',
'embedding_mode': 'word2vec',
'embedding_size': 100,
'hidden_size': 512,
'joint_mode': 'hierarchical',
'level_projection_size': 128,
'lr': 0.001,
'lr_scheduler_factor': 0.9,
'lr_scheduler_patience': 5,
'main_metric': 'micro_f1',
'max_seq_length': 4000,
'metric_level': 1,
'min_seq_length': -1,
'min_word_frequency': -1,
'mode': 'static',
'model': <class 'src.models.rnn.RNN'>,
'multilabel': 1,
'n_epoch': 50,
'n_layers': 1,
'optimiser': 'adamw',
'patience': 5,
'penalisation_coeff': 0.01,
'problem_name': 'mimic-iii_2_full',
'r': -1,
'resume_training': False,
'rnn_model': 'LSTM',
'save_best_model': 1,
'save_results': 1,
'save_results_on_train': False,
'shuffle_data': 1,
'use_last_hidden_state': 0,
'use_lr_scheduler': 1,
'use_regularisation': False,
'weight_decay': 0}
16:27:46 INFO Preparing the vocab
16:28:19 INFO Saved vocab and data to files
16:28:19 INFO Using cuda:1
16:28:19 INFO # levels: 2
16:28:19 INFO # labels at level 0: 1167
16:28:19 INFO # labels at level 1: 8929
16:28:19 INFO 47719.1631.3372
16:28:34 INFO Saved dataset path: cached_data/mimic-iii_2_full/4895ab78f2519b9330f8b64596585d7a.data.pkl
16:29:45 INFO 47719 instances with 70370595 tokens, Level_0 with 1158 labels, Level_1 with 8692 labels in the train dataset
16:29:45 INFO 1631 instances with 2890443 tokens, Level_0 with 738 labels, Level_1 with 3012 labels in the valid dataset
16:29:45 INFO 3372 instances with 6008978 tokens, Level_0 with 853 labels, Level_1 with 4085 labels in the test dataset
16:29:45 INFO Training epoch #1
17:58:55 INFO Learning rate at epoch #1: 0.001
17:58:55 INFO Loss on Train at epoch #1: 0.01475, micro_f1 on Valid: 0.02883
17:58:55 INFO [NEW BEST] (level_1) micro_f1 on Valid set: 0.02883
17:58:55 INFO Results on Valid set at epoch #1 with Averaged Loss 0.01446
17:58:55 INFO ======== Results at level_0 ========
17:58:55 INFO Results on Valid set at epoch #1 with Loss 0.04842:
[MICRO] accuracy: 0.01967 auc: 0.92422 precision: 0.31572 recall: 0.02055 f1: 0.03859 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 0.00027 auc: 0.47509 precision: 0.00027 recall: 0.00086 f1: 0.00041 P@1: 0.31637 P@5: 0.30546 P@8: 0.32442 P@10: 0.31343 P@15: 0.28567
17:58:55 INFO ======== Results at level_1 ========
17:58:55 INFO Results on Valid set at epoch #1 with Loss 0.00993:
[MICRO] accuracy: 0.01463 auc: 0.93636 precision: 0.24493 recall: 0.01532 f1: 0.02883 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 4e-05 auc: 0.47604 precision: 7e-05 recall: 0.00012 f1: 9e-05 P@1: 0.24893 P@5: 0.23495 P@8: 0.22854 P@10: 0.20815 P@15: 0.17613
17:59:07 INFO Training epoch #2
19:29:29 INFO Learning rate at epoch #2: 0.001
19:29:29 INFO Loss on Train at epoch #2: 0.01283, micro_f1 on Valid: 0.0
19:29:29 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
19:29:29 INFO Early stopping: 1/6
19:29:45 INFO Training epoch #3
21:00:44 INFO Learning rate at epoch #3: 0.001
21:00:44 INFO Loss on Train at epoch #3: 0.01276, micro_f1 on Valid: 0.02193
21:00:44 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
21:00:44 INFO Early stopping: 2/6
21:00:57 INFO Training epoch #4
22:33:08 INFO Learning rate at epoch #4: 0.001
22:33:08 INFO Loss on Train at epoch #4: 0.01273, micro_f1 on Valid: 0.0
22:33:08 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
22:33:08 INFO Early stopping: 3/6
22:33:18 INFO Training epoch #5
23:59:54 INFO Learning rate at epoch #5: 0.001
23:59:54 INFO Loss on Train at epoch #5: 0.01271, micro_f1 on Valid: 0.0
23:59:54 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
23:59:54 INFO Early stopping: 4/6
00:00:14 INFO Training epoch #6
01:24:16 INFO Learning rate at epoch #6: 0.001
01:24:16 INFO Loss on Train at epoch #6: 0.01269, micro_f1 on Valid: 0.0
01:24:16 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
01:24:16 INFO Early stopping: 5/6
01:24:29 INFO Training epoch #7
02:48:20 INFO Learning rate at epoch #7: 0.0009000000000000001
02:48:20 INFO Loss on Train at epoch #7: 0.01268, micro_f1 on Valid: 0.02239
02:48:20 INFO [CURRENT BEST] (level_1) micro_f1 on Valid set: 0.02883
02:48:20 INFO Early stopping: 6/6
02:48:44 WARNING Early stopped on Valid set!
02:48:44 INFO =================== BEST ===================
02:48:44 INFO Results on Valid set at epoch #1 with Averaged Loss 0.01446
02:48:44 INFO ======== Results at level_0 ========
02:48:44 INFO Results on Valid set at epoch #1 with Loss 0.04842:
[MICRO] accuracy: 0.01967 auc: 0.92422 precision: 0.31572 recall: 0.02055 f1: 0.03859 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 0.00027 auc: 0.47509 precision: 0.00027 recall: 0.00086 f1: 0.00041 P@1: 0.31637 P@5: 0.30546 P@8: 0.32442 P@10: 0.31343 P@15: 0.28567
02:48:44 INFO ======== Results at level_1 ========
02:48:44 INFO Results on Valid set at epoch #1 with Loss 0.00993:
[MICRO] accuracy: 0.01463 auc: 0.93636 precision: 0.24493 recall: 0.01532 f1: 0.02883 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 4e-05 auc: 0.47604 precision: 7e-05 recall: 0.00012 f1: 9e-05 P@1: 0.24893 P@5: 0.23495 P@8: 0.22854 P@10: 0.20815 P@15: 0.17613
02:48:44 INFO Results on Test set at epoch #1 with Averaged Loss 0.01484
02:48:44 INFO ======== Results at level_0 ========
02:48:44 INFO Results on Test set at epoch #1 with Loss 0.0495:
[MICRO] accuracy: 0.02047 auc: 0.92331 precision: 0.3363 recall: 0.02133 f1: 0.04011 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 0.00029 auc: 0.4488 precision: 0.00029 recall: 0.00086 f1: 0.00043 P@1: 0.3363 P@5: 0.30919 P@8: 0.33278 P@10: 0.3196 P@15: 0.29187
02:48:44 INFO ======== Results at level_1 ========
02:48:44 INFO Results on Test set at epoch #1 with Loss 0.01023:
[MICRO] accuracy: 0.01676 auc: 0.93503 precision: 0.26517 recall: 0.01757 f1: 0.03296 P@1: 0 P@5: 0 P@8: 0 P@10: 0 P@15: 0
[MACRO] accuracy: 4e-05 auc: 0.47837 precision: 7e-05 recall: 0.00013 f1: 9e-05 P@1: 0.26898 P@5: 0.24614 P@8: 0.23458 P@10: 0.21059 P@15: 0.1828
02:48:44 INFO => loading best model 'checkpoints/mimic-iii_2_full/RNN_LSTM_1_512.static.label.0.001.0.3_72df8e44d8921dd19f07bab290d6a868/best_model.pkl'