With nkululeko since version 1.1.3, there is the possibility to run the demo module on pre-loaded models, i.e. feature extractors.
You can simply run the module feature_demo even without any initialization file, for example if you want to test the output of the agender model, you could call python like this (getting input from the local microphone):

 python -m nkululeko.feature_demo --model agender --mic

and get the output:

No config file specified, using default settings

================================================================================
MICROPHONE RECORDING
================================================================================
Recording NOW for 5 seconds - speak clearly!
Recording finished!
Audio saved to temporary file: /tmp/tmpht4rf3mr.wav

Processing 1 file(s)...
DEBUG: feature_demo: running feature extraction demo, nkululeko version 1.1.3
DEBUG: feature_demo: using model: agender

Initializing agender model...
(This may take a while on first run - downloading models...)
  Loading agender_agender model (this may download the model on first use)...
DEBUG: featureset: value for n_jobs is not found, using default: 8
DEBUG: featureset: value for agender.model is not found, using default: ./audmodel_agender/
DEBUG: featureset: initialized agender model
Model initialized successfully!
[1/1] Extracting features from: /tmp/tmpht4rf3mr.wav
Feature dimension: 4

Features saved to features_output.csv

================================================================================
FEATURE EXTRACTION SUMMARY
================================================================================
Total files processed: 1
Feature dimension: 4
Output shape: (1, 4)

First few rows:
                        feat_0    feat_1    feat_2    feat_3
/tmp/tmpht4rf3mr.wav  0.495758 -0.765581  6.437115 -4.332793
Cleaned up temporary recording file

DONE

In this case the output signifies the logits for age, female, male and child.

If you want to see all possible models, simply call

 python -m nkululeko.feature_demo --help

With nkululeko since version 0.94 (aleatoric) uncertainty, i.e. the confidence of the model, is explicitly visualized. You simply find a plot in the image folder after running an experiment, like so:

You see the distribution for true vs. false predictions wrt. uncertainty, i.e. in this case this worked out quite well (because less uncertain prediction are usually correct).

The approach is described in our paper Uncertainty-Based Ensemble Learning For Speech Classification

You can use this to tweak your results if you specify an uncertainty-threshold, i.e. you refuse to predict sample that are above some threshold:

[PLOT]
uncertainty_thresshold = .4

You will than get additionally a confusion plot that only takes the selected samples into account.

This might feel like cheating, but especially in critical use cases it might be better to deliver not prediction than a wrong one.

As described in this previous post, features scaling can be quite important in machine learning.

With nkululeko since version 0.97 you have a multitude if scaling methods at hand.

You simply state in the config:

[FEATS]
scale = xxx

For xxx you specify the scaling methods are

• standard: z-transformation (mean of 0 and std of 1) based on the training set
  • robust: robust scaler
• speaker: like standard but based on individual speaker sets (also for the test)
• bins: convert feature values into 0, .5 and 1 (for low, mid and high)
• minmax: rescales the data set such that all feature values are in the range [0, 1]
• maxabs: similar to MinMaxScaler except that the values are mapped across several ranges depending on whether negative OR positive values are present
• normalizer: scales each sample (row) individually to have unit norm (e.g., L2 norm)
• powertransformer: applies a power transformation to each feature to make the data more Gaussian-like in order to stabilize variance and minimize skewness
• quantiletransformer: applies a non-linear transformation such that the probability density function of each feature will be mapped to a uniform or Gaussian distribution (range [0, 1])

With nkululeko since version 0.96 you there are linguistic feature extractors, i.e. using the text of the spoḱen words as input.

Of course you can combine them with acoustic features and use any fitting model architecture with it.

[EXP]
# optional: language for linguistics
language = de

[DATA]
data = ../mydata
# the linguistic feature extractors require a column named "text"
# example, perhaps not needed!
data.col_names = {"transcription":"text"}

[FEAT]
# combine linguistic bert features with acoustic open smile features
type = ['bert', 'os']

[MODEL]
type = xgb

With nkululeko since version 0.95.9 you can use google translate to translate your data automatically.

Simply set the language (default is en) in the PREDICT section and a prediction target translation like this:

[EXP]
# optional
language = de 
[PREDICT]
targets = ['translation']
# optional
target_language = en 

and then run the module:

python -m nkululeko.predict --config my_conf.ini.