Mismatch between confmatrix dim and labels length about nkululeko

[felixbur]

couldn't replicate yet

[felixbur]

but tess performs nicely as a training for emoodb

[bagustris]

This error actually happens if we mix the filter (i.e., labels) with other filters, e.g,

check_size = 1000
min_duration_of_samples = 2

Will close this issue since no further action is required.

[bagustris]

I am re-opensing this issue.
Simple test to reproduce,

python -m nkululeko.nkululeko --config tests/exp_emodb_audmodel_xgb.ini

Or,

python -m nkululeko.nkululeko --config tests/exp_emodb_os_knn.ini

Output

DEBUG nkululeko: running exp_emodb_audmodel from config tests/exp_emodb_audmodel_xgb.ini, nkululeko version 0.83.0
DEBUG experiment: value for type not found, using default: audformat
DEBUG dataset: emodb: loading from ./data/emodb/emodb
DEBUG dataset: emodb: loading tables: []
DEBUG dataset: value for files_tables not found, using default: ['files']
DEBUG dataset: value for label not found, using default: emotion
DEBUG dataset: value for target_tables not found, using default: ['emotion']
DEBUG dataset: emodb: loaded with 535 samples: got targets: True, got speakers: True (10), got sexes: True
DEBUG datafilter: emodb: limited samples to 50, reduced samples from 535 to 50
DEBUG experiment: target: emotion
DEBUG experiment: Target labels (from config): ['angry', 'happy', 'neutral', 'sad']
DEBUG experiment: loaded databases emodb
DEBUG dataset: splitting database emodb with strategy random
DEBUG dataset: value for test_size not found, using default: 20
DEBUG dataset: emodb: [40/10] samples in train/test
DEBUG dataset: emodb: 10 samples in test and 40 samples in train
DEBUG dataset: emodb: mapped {'anger': 'angry', 'happiness': 'happy', 'sadness': 'sad', 'neutral': 'neutral'}
DEBUG dataset: emodb: mapped {'anger': 'angry', 'happiness': 'happy', 'sadness': 'sad', 'neutral': 'neutral'}
DEBUG experiment: value for type not found, using default: dummy
DEBUG experiment: Categories test (nd.array): ['neutral' 'happy']
DEBUG experiment: Categories train (nd.array): ['neutral' 'angry' 'sad' 'happy']
DEBUG experiment: 4 speakers in test and 7 speakers in train
DEBUG nkululeko: train shape : (25, 8), test shape:(6, 8)
DEBUG featureset: extracting audmodel embeddings, this might take a while...
DEBUG featureset: value for aud.model not found, using default: ./audmodel/
DEBUG featureset: value for device not found, using default: cuda
DEBUG experiment: All features: train shape : (25, 1024), test shape:(6, 1024)                      
DEBUG experiment: scaler: standard
DEBUG scaler: scaling features based on training set
DEBUG runmanager: run 0
DEBUG modelrunner: run: 0 epoch: 0: result: test: 0.400 UAR
DEBUG modelrunner: plotting confusion matrix to emodb_xgb_audmodel_scale-standard_0_000_cnf
ERROR reporter: mismatch between confmatrix dim (3) and labels length (4: ['angry', 'happy', 'neutral', 'sad'])

See my comment above. Maybe it is related to limitation config in database (e.g., db.limit _samples).

[bagustris]

My bad,

This happens when using the same name (output dir) for different experiments (e.g. in test files).

[felixbur]

sure, it's because there are labels in the train that are not in the test set.
But i think you got a point, this should not be a problem.