My Experience with Optuna

Part 1: Introduction

• PyTorch example code

where you will find the following functions that define the hyper-parameters

1. trial.suggest_int(“n_layers”, 1, 3)
2. trial.suggest_categorical(“optimizer”, [“Adam”, “RMSprop”])
3. trial.suggest_float(“lr”, 1e-5, 1e-1, log=True)

• In Optuna there are three terminologies:

1. objective: objective function that you want to optimize
2. trial: a single call of the objective function
3. study: an optimization session, which is a set of trials
4. parameters: a variable whose value is to be optimized

Reference

• Optuna Examples