Nkululeko: segmenting a database

Segmenting a database means to split the audio samples of a database into smaller segments or chunks. With speech data this is usually done on the basis of VAD, aka voice activity detection, meaning that the pauses between speech in the audio samples are used as segment borders.

The reason for segmenting could be to label the data with something that would not last over the whole sample, e.g. emotional state.
Another motivation to segment audio data might be that the acoustic features are targeted at a specific stretch of audio, e.g. 3-5 seconds long.

Within nkululeko this would be done with the segment module, which is currently based on the silero software.

You simply call your experiment configuration with the segment module, and the train, test set or both will be segmented.
The advantage is, that you can use all filters on your data that might make sense beforehand, for example with the android corpus, only the reading task samples are not segmented.
You can select them like so:

[DATA]
filter = [['task', 'reading']]

and then call the segment module:

python -m nkululeko.segment --config my_conf.ini

The output is a new database file in CSV format.

If you want, you can specify if only the training, or test split, or both should be segmented, as well as the string that is added to the name of the resulting csv file (the name per default consists of the database names):

[SEGMENT]
# name postfix
target = _segmented
# which model to use
method = silero
# which split: train, test or all (both)
sample_selection = all
# the minimum lenght of rest-samples (in seconds)
min_length = 2
# the maximum length of segments, longer ones are cut here.  (in seconds)
max_length = 10

4 thoughts on “Nkululeko: segmenting a database”

  1. How is segmenting a database typically performed in the context of audio samples, specifically speech data? What is the role of voice activity detection (VAD) in this process, and how are the pauses between speech used to determine segment borders?

    1. >how are the pauses between speech used to determine segment borders?

      they are cut out:
      [seg1][pause][seg2]

  2. I think this segment module is broken. Several functions are undefined: read_audio, get_speech_sampling_rate, vad_model, SAMPLING_RATE, segmented_index, etc.

Leave a Reply

Your email address will not be published. Required fields are marked *