Here's one idea how to find the optimal values for 2 layers of an MLP net with nkululeko:
- store your meta-parameters in arrays
- loop over them and initialize an experiment each time
- keep the experiment name but change your parameters and the plot name
- this way you can re-use your extracted features and do not get your harddisk cluttered.
Here's some python code to illustrate this idea:
def main(config_file):
# load one configuration per experiment
config = configparser.ConfigParser()
config.read(config_file)
util = Util()
l1s = [32, 64, 128]
l2s = [16, 32, 64]
for l1 in l1s:
for l2 in l2s:
# create a new experiment
expr = exp.Experiment(config)
plotname = f'{util.get_exp_name()}_{l1}_{l2}'
util.set_config_val('PLOT', 'name', plotname)
print(f'running {expr.name} with layers {l1} and {l2}')
layers = {'l1':l1, 'l2':l2}
util.set_config_val('MODEL', 'layers', layers)
# load the data
expr.load_datasets()
# split into train and test
expr.fill_train_and_tests()
util.debug(f'train shape : {expr.df_train.shape}, test shape:{expr.df_test.shape}')
# extract features
expr.extract_feats()
util.debug(f'train feats shape : {expr.feats_train.df.shape}, test feats shape:{expr.feats_test.df.shape}')
# initialize a run manager
expr.init_runmanager()
# run the experiment
expr.run()
print('DONE')
Keep in mind though that meta parameter optimization like done here is in itself a learning problem. It is usually not feasible to systematically try out all combinations of possible values and thus some kind of stochastic approach is preferable.