If you just want to see how your data distributes on the target and speaker gender, you can do a value_counts plot with the explore module
In your config, you would specify like this:
[EXPL]
# all samples, or only test or train split?
sample_selection = all
# activate the plot
value_counts = Trueand then, run this with the explore module:
python -m nkululeko.explore --config myconfig.iniThe result looks similar to this:
It can be very interesting to reduce the dimensionality of your acoustic or learned features to two or three dimensions and then color the single samples features with the label.
Nkululeko supports three different ways to reduce the dimensionality:
To do this, you simply state your data and features as usual. The approaches you want to use can be set in the scatter field of the EXPL section:
[EXPL]
scatter = ['umap', 'tsne', 'pca'](of course you don't have to use all) and then call the explore interface
python -m nkululeko.explore --config myconfig.iniThe images appear in the image folder of your experiment and might look like this (all from the same data):
PCA
T-SNE
UMAP
There is a multitude of ways to model emotions, and some of them are collected in the EmotionML vacabularies.
Really popular with engineers and non-psychologists are two approaches:
The emotion cube maps the emotional categories to a three dimensional space:
To do data augmentation with Nkululeko, you can use the augment interface.
In the DATA section of your configuration file, you specify the name of the output list of files like so
[DATA]
augment = my_augmentations.csvand then call the interface:
python - nkululeko.augment --config myconfig.iniCurrently, Nkululeko simply uses the augmentations that are specified as a demo in the audiomentations documentation, i.e.:
self.audioment = Compose([
AddGaussianNoise(min_amplitude=0.001, max_amplitude=0.015, p=0.5),
TimeStretch(min_rate=0.8, max_rate=1.25, p=0.5),
PitchShift(min_semitones=-4, max_semitones=4, p=0.5),
Shift(min_fraction=-0.5, max_fraction=0.5, p=0.5),
])These manipulations are applied randomly to your training set.
You should find the augmented files in the storage folder of the result folder of your experiment and could listen to them there.
Once you augmentations have been processed, you can add them to the training in a new experiment:
[DATA]
databases = ['original data', 'augment']
augment = my_augmentations.csv
augment.type = csv
augment.absolute_path = True
augment.split_strategy = trainSupervised vs. unsupervised
means the distinction whether your training data is annotated (or labeled) with respect to your task. An example: If you want to build a machine learner for human age estimation based on speech, you might give an algorithm a lot of examples of human speech annotated with the age of the person. This would be your training data and the approach would be supervised (by the age annotations). With unsupervised learning, you would give an algorithm simply a lot of human speech data and might ask it to cluster the data, based on differences. And might hope that the resulting clusters coincide with age.
Nkululeko exercise
-> Nkululeko: install the Berlin Emodb
This database contains examples of labels:
Since version 0.40, Nkululeko can now show the best performing X acoustic features according to some model.
There is a new section call EXPL (short for exploration), and you could state
[EXPL]
model = tree
sample_num = 15in your config file, and then run the exploration module like this:
python -m nkululeko.explore --config my_config.iniThe resulting list will then appear in the result folder and a barplot image in the image folder.
As shown in this post, with Nkululeko you can select only specific features from your features sets by specifying them in the [FEAT] section:
[FEATS]
features = ['JitterPCA', 'meanF0Hz', 'hld_sylRate']What you can also do, is plotting them per category (only for classification), by specifying in the PLOT section if you would like that for all samples or only test or train samples:
[EXPL]
# turn it on
feature_distributions = True
# use only training samples
sample_selection = train
# only plot the 5 most important features
max_feats = 5 You would have to call nkululeko with the explore interface:
python -m nkululeko.explore --config <myConfig.ini>The image file is in the image folder and should look similar to this:
There are three ways to predict a number of samples:
If you want to save the predictions of an experiment for later use, you can do so by stating in the EXP section
[EXP]
save_test = ./my_saved_test_predictions.csvThe output format is CSV, comma seperated values.
Alternatively, you can test an existing database against the best model you trained before, by stating the databases as tests in the DATA section:
[DATA]
tests = ['my_testdb']
my_testdb = /mypath/my_testdb
...and then calling Nkululeko's test module
python -m nkululeko.test --config mycoonfg.ini --outfile myresults.csvRun the demo module simply for a set of files:
python -m nkululeko.demo --config mycoonfg.ini --list my_filelist.txtwrite your
like in the nkululeko project
When you have a new version, you can test it with
pip install .and, when happy,
python -m build will generate the dist folder
twine upload dist/* will upload to pypi server
If you want to develop your package, you can run
python setup.py develop"Normalizing" or scaling feature values means to shift them to a common range, or distribution with same mean and standard deviation (also called z-transformation).
You would do that for several reasons:
Mind that you shouldn't use your test set for normalization as it really only should be used for test and is supposed to be unknown. That's why you should compute your normalization parameters on the training set, you can then use them to normalize/scale the test.