Since version 0.27.0, Nululeko has a concept for a test set, despite train and dev set.

Let's recap the concept of train/dev/test splits:

• train is used to train a supervised model
• dev is a set to evaluate this model, i.e. know when it is a good model (that doesn't overfit)
• test is a set to be used ONLY once: for the real use of the model. If you would use the test as a dev set, you can't be sure if you're not overfitting again (because you used the dev set to adjust the meta parameters of your model).

So, in order to evaluate a third dataset ( beneath train and dev) you set a label_data entry in the configuration [DATA] section like so:

[DATA]
...
label_data = emovo
label_result = my_label_result.csv

and then run the experiment.