My Experience with Optuna

May 4, 2021


Part 1: Introduction

  • PyTorch example code

where you will find the following functions that define the hyper-parameters

  1. trial.suggest_int(“n_layers”, 1, 3)
  2. trial.suggest_categorical(“optimizer”, [“Adam”, “RMSprop”])
  3. trial.suggest_float(“lr”, 1e-5, 1e-1, log=True)
  • In Optuna there are three terminologies:
  1. objective: objective function that you want to optimize
  2. trial: a single call of the objective function
  3. study: an optimization session, which is a set of trials
  4. parameters: a variable whose value is to be optimized