Sometimes you will want to run several experiments without the need to manually start them one after the other, e.g. if you want to run them over night.
This post shows you one way how to do this.

You need two files:
Examples of these files are part of the Nkululeko distribution.

Since version 0.82.0, there's a module named nkuluflag to call nkululeko with many command-line options in addition to the config file (which you still need as a basis).

The configuration file

A Nkululeko config file with the constant values for all experiments (to be adapted to your needs and paths)

[EXP]
root = ./
name = exp
runs = 1
epochs = 1
[DATA]
root_folders = ../data_roots.ini
databases = ['mydata']
target = mytarget
labels = ['label1', 'label2']
[FEATS]
scale = standard
[MODEL]
C_val = .001

The script to specify and run all experiments

Lastly, you need a script to start and specify the experiments, here's an example that combines four classifiers and eight feature sets, resulting in 32 experiments, let's call it do_experiments.py:

import os

classifiers = [
    {"--model": "mlp", "--layers": "\"{'l1':64,'l2':16}\"", "--epochs": 100},
    { "--model": "mlp",
        "--layers": "\"{'l1':128,'l2':64,'l3':16}\"",
        "--learning_rate": ".01",
        "--drop": ".3",
        "--epochs": 100,
    },
    {"--model": "xgb"},
    {"--model": "svm", "C_val": 10},

features = [
    {'--feat': 'os'},
    {'--feat': 'os', 
    '--set': 'ComParE_2016',
    },
    {'--feat': 'wavlm'},
    {'--feat': 'audmodel'},
    {'--feat': 'hubert'},
    {'--feat': 'trill'},
    {'--feat': 'whisper'},
    {'--feat': 'wav2vec'},
]

for c in classifiers:
    for f in features:
        cmd = 'python -m nkululeko.nkuluflag --config myconf.ini  '
        for item in c:
            cmd += f'{item} {c[item]} '
        for item in f:
            cmd += f'{item} {f[item]} '
        print(cmd)
        os.system(cmd)

You can then simply call you script with python:

python do_experiments.py