There is a multitude of ways to model emotions, and some of them are collected in the EmotionML vacabularies.
Really popular with engineers and non-psychologists are two approaches:
The emotion cube maps the emotional categories to a three dimensional space:
To do data augmentation with Nkululeko, you can use the augment interface.
In the DATA section of your configuration file, you specify the method and name of the output list of file
[DATA]
# select the samples to augment: either train, test, or all
augment = train
# file name to store the augmented data (can then be added to training)
augment_result = augment.csv
or, if you want to use random-splicing:
[DATA]
# random_splice: select the samples to be random spliced: either train, test, or all
random_splice = train
# random_splice_result: file name to store the random spliced data (can then be added to training)
random_splice_result = random_spliced.csvand then call the interface:
python -m nkululeko.augment --config myconfig.iniCurrently, apart from random-splicing, Nkululeko simply uses the augmentations that are specified as a demo in the audiomentations documentation, i.e.:
self.audioment = Compose([
AddGaussianNoise(min_amplitude=0.001, max_amplitude=0.015, p=0.5),
TimeStretch(min_rate=0.8, max_rate=1.25, p=0.5),
PitchShift(min_semitones=-4, max_semitones=4, p=0.5),
Shift(min_fraction=-0.5, max_fraction=0.5, p=0.5),
])These manipulations are applied randomly to your training set.
You should find the augmented files in the storage folder of the result folder of your experiment and could listen to them there.
Once you augmentations have been processed, you can add them to the training in a new experiment:
[DATA]
databases = ['original data', 'augment']
augment = my_augmentations.csv
augment.type = csv
augment.absolute_path = True
augment.split_strategy = trainSupervised vs. unsupervised
means the distinction whether your training data is annotated (or labeled) with respect to your task. An example: If you want to build a machine learner for human age estimation based on speech, you might give an algorithm a lot of examples of human speech annotated with the age of the person. This would be your training data and the approach would be supervised (by the age annotations). With unsupervised learning, you would give an algorithm simply a lot of human speech data and might ask it to cluster the data, based on differences. And might hope that the resulting clusters coincide with age.
Nkululeko exercise
-> Nkululeko: install the Berlin Emodb
This database contains examples of labels: